Note: This is test shard 1 of 3.
[==========] Running 2 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 2 tests from CatalogManagerConfigurations/MasterStressTest
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/0
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260924 00:20:43.010242 10257 test_util.cc:268] Using random seed: -1845439417
2026-09-24T00:20:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:20:43Z Disabled control of system clock
I20260924 00:20:43.058449 10257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu
/tmp/dist-test-taskko7HPF/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.4.126:34479
--webserver_interface=127.10.4.126
--webserver_port=0
--builtin_ntp_servers=127.10.4.84:38335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.4.126:34479,127.10.4.125:34487,127.10.4.124:35345
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260924 00:20:43.338598 10266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:20:43.339136 10266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:20:43.339531 10266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:20:43.369992 10266 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:20:43.370276 10266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:20:43.370486 10266 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:20:43.370683 10266 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:20:43.404871 10266 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.4.84:38335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-0/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.10.4.126:34479,127.10.4.125:34487,127.10.4.124:35345
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.4.126:34479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.4.126
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:32:10 UTC on ccycloud.quasar-wfxeao.root.comops.site
build id 4668917
TSAN enabled
I20260924 00:20:43.406113 10266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:20:43.407650 10266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:20:43.423938 10273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:20:43.424016 10272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:20:43.425349 10275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:20:43.425500 10266 server_base.cc:1034] running on GCE node
I20260924 00:20:44.517364 10266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:20:44.520264 10266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:20:44.521679 10266 hybrid_clock.cc:648] HybridClock initialized: now 1790209244521618 us; error 78 us; skew 500 ppm
I20260924 00:20:44.522441 10266 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 00:20:44.528698 10266 webserver.cc:460] Webserver started at http://127.10.4.126:40661/ using document root <none> and password file <none>
I20260924 00:20:44.529599 10266 fs_manager.cc:359] Metadata directory not provided
I20260924 00:20:44.529776 10266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:20:44.530169 10266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:20:44.534886 10266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-0/data/instance:
uuid: "1d68651913494ce8afee3c136b2da9cc"
format_stamp: "Formatted at 2026-09-24 00:20:44 on dist-test-slave-g3nd"
I20260924 00:20:44.535938 10266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-0/wal/instance:
uuid: "1d68651913494ce8afee3c136b2da9cc"
format_stamp: "Formatted at 2026-09-24 00:20:44 on dist-test-slave-g3nd"
I20260924 00:20:44.542985 10266 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.001s
I20260924 00:20:44.548458 10282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:20:44.549595 10266 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260924 00:20:44.549916 10266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-0/data,/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-0/wal
uuid: "1d68651913494ce8afee3c136b2da9cc"
format_stamp: "Formatted at 2026-09-24 00:20:44 on dist-test-slave-g3nd"
I20260924 00:20:44.550220 10266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:20:44.602301 10266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:20:44.603760 10266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:20:44.604187 10266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:20:44.669744 10266 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.126:34479
I20260924 00:20:44.669818 10333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.4.126:34479 every 8 connection(s)
I20260924 00:20:44.672310 10266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-0/data/info.pb
I20260924 00:20:44.677512 10334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:44.678049 10257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu as pid 10266
I20260924 00:20:44.678542 10257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-0/wal/instance
I20260924 00:20:44.683676 10257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu
/tmp/dist-test-taskko7HPF/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.4.125:34487
--webserver_interface=127.10.4.125
--webserver_port=0
--builtin_ntp_servers=127.10.4.84:38335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.4.126:34479,127.10.4.125:34487,127.10.4.124:35345
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260924 00:20:44.691674 10334 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:20:44.718112 10334 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } has no permanent_uuid. Determining permanent_uuid...
W20260924 00:20:44.720377 10286 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.125:34487: connect: Connection refused (error 111)
W20260924 00:20:44.724005 10334 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.125:34487: Network error: Client connection negotiation failed: client connection to 127.10.4.125:34487: connect: Connection refused (error 111)
I20260924 00:20:44.773442 10334 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } attempt: 1
W20260924 00:20:44.776998 10334 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.125:34487: Network error: Client connection negotiation failed: client connection to 127.10.4.125:34487: connect: Connection refused (error 111)
I20260924 00:20:44.845377 10334 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } attempt: 2
W20260924 00:20:44.848930 10334 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.125:34487: Network error: Client connection negotiation failed: client connection to 127.10.4.125:34487: connect: Connection refused (error 111)
I20260924 00:20:44.940363 10334 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } attempt: 3
W20260924 00:20:44.944237 10334 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.125:34487: Network error: Client connection negotiation failed: client connection to 127.10.4.125:34487: connect: Connection refused (error 111)
W20260924 00:20:44.977243 10336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:20:44.977831 10336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:20:44.978272 10336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:20:45.008925 10336 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:20:45.009258 10336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:20:45.009519 10336 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:20:45.009779 10336 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:20:45.043759 10336 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.4.84:38335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-1/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.10.4.126:34479,127.10.4.125:34487,127.10.4.124:35345
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.4.125:34487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.4.125
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:32:10 UTC on ccycloud.quasar-wfxeao.root.comops.site
build id 4668917
TSAN enabled
I20260924 00:20:45.044994 10336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:20:45.046574 10336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:20:45.063370 10346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:20:45.087633 10334 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } attempt: 4
W20260924 00:20:45.091307 10334 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.125:34487: Network error: Client connection negotiation failed: client connection to 127.10.4.125:34487: connect: Connection refused (error 111)
I20260924 00:20:45.390677 10334 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } attempt: 5
W20260924 00:20:45.394587 10334 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.125:34487: Network error: Client connection negotiation failed: client connection to 127.10.4.125:34487: connect: Connection refused (error 111)
I20260924 00:20:45.941978 10334 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } attempt: 6
W20260924 00:20:45.953150 10334 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.125:34487: Network error: Client connection negotiation failed: client connection to 127.10.4.125:34487: connect: Connection refused (error 111)
W20260924 00:20:45.063416 10345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:20:45.063874 10348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:20:45.065645 10336 server_base.cc:1034] running on GCE node
I20260924 00:20:46.163228 10336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:20:46.166074 10336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:20:46.167471 10336 hybrid_clock.cc:648] HybridClock initialized: now 1790209246167411 us; error 74 us; skew 500 ppm
I20260924 00:20:46.168294 10336 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 00:20:46.180111 10336 webserver.cc:460] Webserver started at http://127.10.4.125:45651/ using document root <none> and password file <none>
I20260924 00:20:46.181067 10336 fs_manager.cc:359] Metadata directory not provided
I20260924 00:20:46.181279 10336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:20:46.181703 10336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:20:46.186424 10336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-1/data/instance:
uuid: "b5aa3debe1cd45e19a6bbd0070913ac3"
format_stamp: "Formatted at 2026-09-24 00:20:46 on dist-test-slave-g3nd"
I20260924 00:20:46.187582 10336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-1/wal/instance:
uuid: "b5aa3debe1cd45e19a6bbd0070913ac3"
format_stamp: "Formatted at 2026-09-24 00:20:46 on dist-test-slave-g3nd"
I20260924 00:20:46.194852 10336 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.003s
I20260924 00:20:46.200219 10357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:20:46.201293 10336 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 00:20:46.201598 10336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-1/data,/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-1/wal
uuid: "b5aa3debe1cd45e19a6bbd0070913ac3"
format_stamp: "Formatted at 2026-09-24 00:20:46 on dist-test-slave-g3nd"
I20260924 00:20:46.201911 10336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:20:46.248430 10336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:20:46.249928 10336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:20:46.250378 10336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:20:46.314900 10336 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.125:34487
I20260924 00:20:46.315013 10408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.4.125:34487 every 8 connection(s)
I20260924 00:20:46.317554 10336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-1/data/info.pb
I20260924 00:20:46.321622 10257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu as pid 10336
I20260924 00:20:46.322091 10257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-1/wal/instance
I20260924 00:20:46.323140 10409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:46.325119 10257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu
/tmp/dist-test-taskko7HPF/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.4.124:35345
--webserver_interface=127.10.4.124
--webserver_port=0
--builtin_ntp_servers=127.10.4.84:38335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.4.126:34479,127.10.4.125:34487,127.10.4.124:35345
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260924 00:20:46.335825 10409 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:20:46.358402 10409 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:20:46.372548 10409 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } has no permanent_uuid. Determining permanent_uuid...
W20260924 00:20:46.374558 10361 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.124:35345: connect: Connection refused (error 111)
W20260924 00:20:46.378203 10409 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.124:35345: Network error: Client connection negotiation failed: client connection to 127.10.4.124:35345: connect: Connection refused (error 111)
I20260924 00:20:46.427758 10409 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } attempt: 1
W20260924 00:20:46.431344 10409 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.124:35345: Network error: Client connection negotiation failed: client connection to 127.10.4.124:35345: connect: Connection refused (error 111)
I20260924 00:20:46.499778 10409 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } attempt: 2
W20260924 00:20:46.503243 10409 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.124:35345: Network error: Client connection negotiation failed: client connection to 127.10.4.124:35345: connect: Connection refused (error 111)
I20260924 00:20:46.594640 10409 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } attempt: 3
W20260924 00:20:46.598219 10409 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.124:35345: Network error: Client connection negotiation failed: client connection to 127.10.4.124:35345: connect: Connection refused (error 111)
W20260924 00:20:46.617404 10411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:20:46.617972 10411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:20:46.618387 10411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:20:46.648247 10411 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:20:46.648566 10411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:20:46.648803 10411 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:20:46.649061 10411 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:20:46.683526 10411 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.4.84:38335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-2/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.10.4.126:34479,127.10.4.125:34487,127.10.4.124:35345
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.4.124:35345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.4.124
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:32:10 UTC on ccycloud.quasar-wfxeao.root.comops.site
build id 4668917
TSAN enabled
I20260924 00:20:46.684710 10411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:20:46.686259 10411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:20:46.703482 10424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:20:46.741603 10409 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } attempt: 4
W20260924 00:20:46.745323 10409 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.124:35345: Network error: Client connection negotiation failed: client connection to 127.10.4.124:35345: connect: Connection refused (error 111)
I20260924 00:20:47.013594 10334 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } attempt: 7
I20260924 00:20:47.028057 10334 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } has no permanent_uuid. Determining permanent_uuid...
W20260924 00:20:47.035270 10334 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.124:35345: Network error: Client connection negotiation failed: client connection to 127.10.4.124:35345: connect: Connection refused (error 111)
I20260924 00:20:47.044772 10409 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } attempt: 5
W20260924 00:20:47.048739 10409 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.124:35345: Network error: Client connection negotiation failed: client connection to 127.10.4.124:35345: connect: Connection refused (error 111)
I20260924 00:20:47.093659 10334 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } attempt: 1
W20260924 00:20:47.097656 10334 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.124:35345: Network error: Client connection negotiation failed: client connection to 127.10.4.124:35345: connect: Connection refused (error 111)
I20260924 00:20:47.179082 10334 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } attempt: 2
W20260924 00:20:47.183281 10334 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.124:35345: Network error: Client connection negotiation failed: client connection to 127.10.4.124:35345: connect: Connection refused (error 111)
I20260924 00:20:47.268689 10334 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } attempt: 3
W20260924 00:20:47.272662 10334 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.124:35345: Network error: Client connection negotiation failed: client connection to 127.10.4.124:35345: connect: Connection refused (error 111)
I20260924 00:20:47.413075 10334 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } attempt: 4
W20260924 00:20:47.417079 10334 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.124:35345: Network error: Client connection negotiation failed: client connection to 127.10.4.124:35345: connect: Connection refused (error 111)
I20260924 00:20:47.596153 10409 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } attempt: 6
W20260924 00:20:47.607079 10409 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.124:35345: Network error: Client connection negotiation failed: client connection to 127.10.4.124:35345: connect: Connection refused (error 111)
I20260924 00:20:47.700461 10334 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } attempt: 5
W20260924 00:20:47.703869 10334 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.124:35345: Network error: Client connection negotiation failed: client connection to 127.10.4.124:35345: connect: Connection refused (error 111)
W20260924 00:20:46.703545 10422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:20:46.703508 10421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:20:46.705019 10411 server_base.cc:1034] running on GCE node
I20260924 00:20:47.803673 10411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:20:47.806236 10411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:20:47.807576 10411 hybrid_clock.cc:648] HybridClock initialized: now 1790209247807536 us; error 58 us; skew 500 ppm
I20260924 00:20:47.808555 10411 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 00:20:47.815037 10411 webserver.cc:460] Webserver started at http://127.10.4.124:35831/ using document root <none> and password file <none>
I20260924 00:20:47.815930 10411 fs_manager.cc:359] Metadata directory not provided
I20260924 00:20:47.816144 10411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:20:47.816582 10411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:20:47.821507 10411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-2/data/instance:
uuid: "f8bd0b3184f84f5f8da10c9c21472f92"
format_stamp: "Formatted at 2026-09-24 00:20:47 on dist-test-slave-g3nd"
I20260924 00:20:47.822573 10411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-2/wal/instance:
uuid: "f8bd0b3184f84f5f8da10c9c21472f92"
format_stamp: "Formatted at 2026-09-24 00:20:47 on dist-test-slave-g3nd"
I20260924 00:20:47.829449 10411 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260924 00:20:47.835155 10436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:20:47.836229 10411 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260924 00:20:47.836624 10411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-2/data,/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-2/wal
uuid: "f8bd0b3184f84f5f8da10c9c21472f92"
format_stamp: "Formatted at 2026-09-24 00:20:47 on dist-test-slave-g3nd"
I20260924 00:20:47.836977 10411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:20:47.870328 10411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:20:47.871760 10411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:20:47.872203 10411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:20:47.938436 10411 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.124:35345
I20260924 00:20:47.938604 10487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.4.124:35345 every 8 connection(s)
I20260924 00:20:47.940990 10411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-2/data/info.pb
I20260924 00:20:47.943854 10257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu as pid 10411
I20260924 00:20:47.944312 10257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-2/wal/instance
I20260924 00:20:47.946695 10488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:47.963178 10488 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:20:47.990873 10488 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:20:48.003332 10488 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:20:48.021958 10488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92: Bootstrap starting.
I20260924 00:20:48.027670 10488 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:48.029340 10488 log.cc:824] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92: Log is configured to *not* fsync() on all Append() calls
I20260924 00:20:48.033948 10488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92: No bootstrap required, opened a new log
I20260924 00:20:48.053610 10488 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } }
I20260924 00:20:48.054273 10488 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:48.054530 10488 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8bd0b3184f84f5f8da10c9c21472f92, State: Initialized, Role: FOLLOWER
I20260924 00:20:48.055230 10488 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } }
I20260924 00:20:48.057487 10499 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } } }
I20260924 00:20:48.058125 10499 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:20:48.058790 10488 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [sys.catalog]: configured and running, proceeding with master startup.
W20260924 00:20:48.083458 10510 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 00:20:48.083794 10510 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 00:20:48.078872 10411 master_runner.cc:186] Error getting master registration for 127.10.4.126:34479: OK, instance_id { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" instance_seqno: 1790209244652081 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 00:20:48.102408 10411 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260924 00:20:48.127053 10442 tablet.cc:2369] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:20:48.256239 10334 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } attempt: 6
I20260924 00:20:48.274405 10334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc: Bootstrap starting.
I20260924 00:20:48.279790 10334 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:48.281570 10334 log.cc:824] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc: Log is configured to *not* fsync() on all Append() calls
I20260924 00:20:48.285930 10334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc: No bootstrap required, opened a new log
I20260924 00:20:48.306825 10334 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } }
I20260924 00:20:48.307448 10334 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:48.307679 10334 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d68651913494ce8afee3c136b2da9cc, State: Initialized, Role: FOLLOWER
I20260924 00:20:48.308401 10334 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } }
I20260924 00:20:48.311170 10514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } } }
I20260924 00:20:48.311977 10514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:20:48.312855 10334 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [sys.catalog]: configured and running, proceeding with master startup.
W20260924 00:20:48.360814 10525 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 00:20:48.361291 10525 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260924 00:20:48.362653 10288 tablet.cc:2369] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:20:48.347898 10266 master_runner.cc:186] Error getting master registration for 127.10.4.125:34487: OK, instance_id { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" instance_seqno: 1790209246297586 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 00:20:48.385942 10266 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:20:48.522961 10514 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:20:48.523501 10514 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } }
I20260924 00:20:48.525923 10514 leader_election.cc:288] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5aa3debe1cd45e19a6bbd0070913ac3 (127.10.4.125:34487), f8bd0b3184f84f5f8da10c9c21472f92 (127.10.4.124:35345)
I20260924 00:20:48.527563 10463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1d68651913494ce8afee3c136b2da9cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" is_pre_election: true
I20260924 00:20:48.528390 10463 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d68651913494ce8afee3c136b2da9cc in term 0.
I20260924 00:20:48.529589 10286 leader_election.cc:302] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d68651913494ce8afee3c136b2da9cc, f8bd0b3184f84f5f8da10c9c21472f92; no voters:
I20260924 00:20:48.530412 10514 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:20:48.530897 10514 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:20:48.531514 10514 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:20:48.543818 10514 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } }
I20260924 00:20:48.550330 10463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1d68651913494ce8afee3c136b2da9cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8bd0b3184f84f5f8da10c9c21472f92"
I20260924 00:20:48.550220 10514 leader_election.cc:288] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [CANDIDATE]: Term 1 election: Requested vote from peers b5aa3debe1cd45e19a6bbd0070913ac3 (127.10.4.125:34487), f8bd0b3184f84f5f8da10c9c21472f92 (127.10.4.124:35345)
I20260924 00:20:48.550925 10463 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:20:48.559993 10463 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d68651913494ce8afee3c136b2da9cc in term 1.
I20260924 00:20:48.561606 10286 leader_election.cc:302] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d68651913494ce8afee3c136b2da9cc, f8bd0b3184f84f5f8da10c9c21472f92; no voters:
I20260924 00:20:48.562579 10514 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:20:48.563894 10514 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 1 LEADER]: Becoming Leader. State: Replica: 1d68651913494ce8afee3c136b2da9cc, State: Running, Role: LEADER
I20260924 00:20:48.564821 10514 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } }
I20260924 00:20:48.571914 10527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d68651913494ce8afee3c136b2da9cc. Latest consensus state: current_term: 1 leader_uuid: "1d68651913494ce8afee3c136b2da9cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } } }
I20260924 00:20:48.572640 10527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [sys.catalog]: This master's current role is: LEADER
I20260924 00:20:48.577479 10529 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:20:48.589350 10529 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:20:48.613343 10463 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 1 FOLLOWER]: Refusing update from remote peer 1d68651913494ce8afee3c136b2da9cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:20:48.615653 10527 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:20:48.651453 10531 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d68651913494ce8afee3c136b2da9cc. Latest consensus state: current_term: 1 leader_uuid: "1d68651913494ce8afee3c136b2da9cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } } }
I20260924 00:20:48.652292 10531 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:20:48.661329 10530 mvcc.cc:205] Tried to move back new op lower bound from 7332697082304032768 to 7332697082131447808. Current Snapshot: MvccSnapshot[applied={T|T < 7332697082304032768}]
I20260924 00:20:48.673738 10531 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1d68651913494ce8afee3c136b2da9cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } } }
I20260924 00:20:48.674456 10531 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:20:48.677000 10409 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } attempt: 7
I20260924 00:20:48.683193 10527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1d68651913494ce8afee3c136b2da9cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } } }
I20260924 00:20:48.683923 10527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [sys.catalog]: This master's current role is: LEADER
I20260924 00:20:48.699163 10529 catalog_manager.cc:1347] Generated new cluster ID: 89ba9572944348c7affb30a26e8aad8d
I20260924 00:20:48.699479 10529 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:20:48.713359 10409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Bootstrap starting.
I20260924 00:20:48.727974 10409 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:48.730485 10409 log.cc:824] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Log is configured to *not* fsync() on all Append() calls
I20260924 00:20:48.753079 10409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: No bootstrap required, opened a new log
I20260924 00:20:48.825304 10529 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:20:48.828205 10529 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:20:48.847287 10409 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } }
I20260924 00:20:48.848145 10409 raft_consensus.cc:374] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:48.848431 10409 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5aa3debe1cd45e19a6bbd0070913ac3, State: Initialized, Role: FOLLOWER
I20260924 00:20:48.851982 10409 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } }
I20260924 00:20:48.864418 10529 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc: Generated new TSK 0
I20260924 00:20:48.870501 10529 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:20:48.869868 10538 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } } }
I20260924 00:20:48.872069 10538 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:20:48.877339 10409 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:20:48.877512 10383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1d68651913494ce8afee3c136b2da9cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3"
I20260924 00:20:48.878288 10383 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:20:48.881286 10384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1d68651913494ce8afee3c136b2da9cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" is_pre_election: true
I20260924 00:20:48.887668 10383 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d68651913494ce8afee3c136b2da9cc in term 1.
I20260924 00:20:48.889012 10382 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [term 1 FOLLOWER]: Refusing update from remote peer 1d68651913494ce8afee3c136b2da9cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:20:48.896166 10527 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:20:48.916887 10257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu
/tmp/dist-test-taskko7HPF/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.4.65:0
--local_ip_for_outbound_sockets=127.10.4.65
--webserver_interface=127.10.4.65
--webserver_port=0
--tserver_master_addrs=127.10.4.126:34479,127.10.4.125:34487,127.10.4.124:35345
--builtin_ntp_servers=127.10.4.84:38335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260924 00:20:48.985931 10538 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d68651913494ce8afee3c136b2da9cc. Latest consensus state: current_term: 1 leader_uuid: "1d68651913494ce8afee3c136b2da9cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } } }
I20260924 00:20:48.986662 10538 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:20:48.987635 10547 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1d68651913494ce8afee3c136b2da9cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } } }
I20260924 00:20:48.988472 10547 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:20:49.057534 10555 catalog_manager.cc:1259] Loaded cluster ID: 89ba9572944348c7affb30a26e8aad8d
I20260924 00:20:49.057806 10555 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: loading cluster ID for follower catalog manager: success
I20260924 00:20:49.065207 10555 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: acquiring CA information for follower catalog manager: success
I20260924 00:20:49.070588 10555 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 00:20:49.089229 10510 catalog_manager.cc:1259] Loaded cluster ID: 89ba9572944348c7affb30a26e8aad8d
I20260924 00:20:49.089589 10510 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92: loading cluster ID for follower catalog manager: success
I20260924 00:20:49.096980 10510 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92: acquiring CA information for follower catalog manager: success
I20260924 00:20:49.108860 10510 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 00:20:49.351609 10527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1d68651913494ce8afee3c136b2da9cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } } }
I20260924 00:20:49.352337 10527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [sys.catalog]: This master's current role is: LEADER
W20260924 00:20:49.452327 10541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:20:49.453068 10541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:20:49.454149 10541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:20:49.508795 10541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:20:49.510406 10541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.4.65
I20260924 00:20:49.570640 10541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.4.84:38335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.4.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.4.65
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.10.4.126:34479,127.10.4.125:34487,127.10.4.124:35345
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.4.65
--log_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:32:10 UTC on ccycloud.quasar-wfxeao.root.comops.site
build id 4668917
TSAN enabled
I20260924 00:20:49.572460 10541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:20:49.574631 10541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:20:49.620927 10564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:20:49.868010 10570 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1d68651913494ce8afee3c136b2da9cc)
I20260924 00:20:49.868013 10569 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1d68651913494ce8afee3c136b2da9cc)
I20260924 00:20:49.868732 10570 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } }
I20260924 00:20:49.868727 10569 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } }
I20260924 00:20:49.871982 10570 leader_election.cc:288] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1d68651913494ce8afee3c136b2da9cc (127.10.4.126:34479), f8bd0b3184f84f5f8da10c9c21472f92 (127.10.4.124:35345)
I20260924 00:20:49.872972 10463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" is_pre_election: true
I20260924 00:20:49.873127 10309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1d68651913494ce8afee3c136b2da9cc" is_pre_election: true
I20260924 00:20:49.873533 10463 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5aa3debe1cd45e19a6bbd0070913ac3 in term 1.
I20260924 00:20:49.874727 10361 leader_election.cc:302] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5aa3debe1cd45e19a6bbd0070913ac3, f8bd0b3184f84f5f8da10c9c21472f92; no voters:
I20260924 00:20:49.875507 10570 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 00:20:49.875833 10570 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1d68651913494ce8afee3c136b2da9cc)
I20260924 00:20:49.876204 10570 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [term 1 FOLLOWER]: Advancing to term 2
I20260924 00:20:49.879220 10309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1d68651913494ce8afee3c136b2da9cc" is_pre_election: true
I20260924 00:20:49.882431 10569 leader_election.cc:288] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1d68651913494ce8afee3c136b2da9cc (127.10.4.126:34479), b5aa3debe1cd45e19a6bbd0070913ac3 (127.10.4.125:34487)
I20260924 00:20:49.882583 10384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" is_pre_election: true
I20260924 00:20:49.882511 10570 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } }
I20260924 00:20:49.893420 10384 raft_consensus.cc:2379] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f8bd0b3184f84f5f8da10c9c21472f92 in current term 2: Already voted for candidate b5aa3debe1cd45e19a6bbd0070913ac3 in this term.
I20260924 00:20:49.896245 10440 leader_election.cc:302] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f8bd0b3184f84f5f8da10c9c21472f92; no voters: 1d68651913494ce8afee3c136b2da9cc, b5aa3debe1cd45e19a6bbd0070913ac3
I20260924 00:20:49.897207 10570 leader_election.cc:288] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [CANDIDATE]: Term 2 election: Requested vote from peers 1d68651913494ce8afee3c136b2da9cc (127.10.4.126:34479), f8bd0b3184f84f5f8da10c9c21472f92 (127.10.4.124:35345)
I20260924 00:20:49.898134 10463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f8bd0b3184f84f5f8da10c9c21472f92"
I20260924 00:20:49.898275 10569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 1 FOLLOWER]: Advancing to term 2
I20260924 00:20:49.905344 10309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1d68651913494ce8afee3c136b2da9cc"
I20260924 00:20:49.908165 10569 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 00:20:49.915359 10463 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5aa3debe1cd45e19a6bbd0070913ac3 in term 2.
I20260924 00:20:49.918934 10361 leader_election.cc:302] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b5aa3debe1cd45e19a6bbd0070913ac3, f8bd0b3184f84f5f8da10c9c21472f92; no voters: 1d68651913494ce8afee3c136b2da9cc
I20260924 00:20:49.920073 10570 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 00:20:49.933861 10570 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [term 2 LEADER]: Becoming Leader. State: Replica: b5aa3debe1cd45e19a6bbd0070913ac3, State: Running, Role: LEADER
I20260924 00:20:49.934955 10570 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } }
I20260924 00:20:49.939635 10571 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5aa3debe1cd45e19a6bbd0070913ac3. Latest consensus state: current_term: 2 leader_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } } }
I20260924 00:20:49.940397 10571 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [sys.catalog]: This master's current role is: LEADER
I20260924 00:20:49.969450 10572 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:20:49.979194 10463 raft_consensus.cc:1226] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 2 FOLLOWER]: Rejecting Update request from peer 1d68651913494ce8afee3c136b2da9cc for earlier term 1. Current term is 2. Ops: []
I20260924 00:20:49.989146 10574 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 00:20:49.990538 10573 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 1 LEADER]: Stepping down as leader of term 1
I20260924 00:20:49.990914 10573 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1d68651913494ce8afee3c136b2da9cc, State: Running, Role: LEADER
I20260924 00:20:49.991730 10573 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } }
I20260924 00:20:49.992735 10573 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 1 FOLLOWER]: Advancing to term 2
I20260924 00:20:50.001238 10572 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:20:50.001582 10384 raft_consensus.cc:1226] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [term 2 LEADER]: Rejecting Update request from peer 1d68651913494ce8afee3c136b2da9cc for earlier term 1. Current term is 2. Ops: []
I20260924 00:20:50.006006 10572 catalog_manager.cc:1259] Loaded cluster ID: 89ba9572944348c7affb30a26e8aad8d
I20260924 00:20:50.013073 10572 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:20:50.018471 10572 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:20:50.037132 10572 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Loaded TSK: 0
I20260924 00:20:50.046303 10572 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:20:50.368151 10525 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 00:20:50.456317 10309 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 2 FOLLOWER]: Refusing update from remote peer b5aa3debe1cd45e19a6bbd0070913ac3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 00:20:50.458484 10571 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 00:20:50.478154 10574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5aa3debe1cd45e19a6bbd0070913ac3. Latest consensus state: current_term: 2 leader_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } } }
I20260924 00:20:50.478977 10574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:20:50.500911 10574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } } }
I20260924 00:20:50.501772 10574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:20:50.516553 10463 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 2 FOLLOWER]: Refusing update from remote peer b5aa3debe1cd45e19a6bbd0070913ac3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 00:20:50.520879 10571 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 00:20:50.525831 10576 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } } }
I20260924 00:20:50.528452 10576 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [sys.catalog]: This master's current role is: LEADER
I20260924 00:20:50.569307 10580 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5aa3debe1cd45e19a6bbd0070913ac3. Latest consensus state: current_term: 2 leader_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } } }
I20260924 00:20:50.570293 10580 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:20:50.570938 10580 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } } }
I20260924 00:20:50.571729 10580 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:20:50.997392 10574 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b5aa3debe1cd45e19a6bbd0070913ac3)
I20260924 00:20:50.997915 10574 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } }
I20260924 00:20:50.999791 10574 leader_election.cc:288] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b5aa3debe1cd45e19a6bbd0070913ac3 (127.10.4.125:34487), f8bd0b3184f84f5f8da10c9c21472f92 (127.10.4.124:35345)
I20260924 00:20:51.002503 10579 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } } }
I20260924 00:20:51.002965 10384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1d68651913494ce8afee3c136b2da9cc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" is_pre_election: true
I20260924 00:20:51.003273 10579 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [sys.catalog]: This master's current role is: LEADER
I20260924 00:20:51.004103 10463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1d68651913494ce8afee3c136b2da9cc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" is_pre_election: true
I20260924 00:20:51.006942 10286 leader_election.cc:302] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1d68651913494ce8afee3c136b2da9cc; no voters: b5aa3debe1cd45e19a6bbd0070913ac3, f8bd0b3184f84f5f8da10c9c21472f92
I20260924 00:20:51.010054 10574 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260924 00:20:49.633455 10567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:20:49.634507 10541 server_base.cc:1034] running on GCE node
W20260924 00:20:49.624840 10563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:20:51.102741 10541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:20:51.105331 10541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:20:51.106833 10541 hybrid_clock.cc:648] HybridClock initialized: now 1790209251106771 us; error 77 us; skew 500 ppm
I20260924 00:20:51.107793 10541 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 00:20:51.114431 10541 webserver.cc:460] Webserver started at http://127.10.4.65:40519/ using document root <none> and password file <none>
I20260924 00:20:51.115553 10541 fs_manager.cc:359] Metadata directory not provided
I20260924 00:20:51.115793 10541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:20:51.116355 10541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:20:51.121567 10541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/data/instance:
uuid: "b10a9082a37b46f09d581944914c1670"
format_stamp: "Formatted at 2026-09-24 00:20:51 on dist-test-slave-g3nd"
I20260924 00:20:51.122712 10541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/instance:
uuid: "b10a9082a37b46f09d581944914c1670"
format_stamp: "Formatted at 2026-09-24 00:20:51 on dist-test-slave-g3nd"
I20260924 00:20:51.130061 10541 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260924 00:20:51.135813 10588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:20:51.136858 10541 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260924 00:20:51.137200 10541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/data,/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal
uuid: "b10a9082a37b46f09d581944914c1670"
format_stamp: "Formatted at 2026-09-24 00:20:51 on dist-test-slave-g3nd"
I20260924 00:20:51.137540 10541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:20:51.188119 10541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:20:51.189723 10541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:20:51.190176 10541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:20:51.193029 10541 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:20:51.197468 10541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:20:51.197654 10541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:20:51.197921 10541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:20:51.198066 10541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:20:51.361326 10541 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.65:33461
I20260924 00:20:51.361416 10700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.4.65:33461 every 8 connection(s)
I20260924 00:20:51.364034 10541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/data/info.pb
I20260924 00:20:51.373191 10257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu as pid 10541
I20260924 00:20:51.373670 10257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/instance
I20260924 00:20:51.392156 10257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu
/tmp/dist-test-taskko7HPF/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.4.66:0
--local_ip_for_outbound_sockets=127.10.4.66
--webserver_interface=127.10.4.66
--webserver_port=0
--tserver_master_addrs=127.10.4.126:34479,127.10.4.125:34487,127.10.4.124:35345
--builtin_ntp_servers=127.10.4.84:38335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260924 00:20:51.431421 10702 heartbeater.cc:347] Connected to a master server at 127.10.4.125:34487
I20260924 00:20:51.431896 10702 heartbeater.cc:464] Registering TS with master...
I20260924 00:20:51.433547 10702 heartbeater.cc:511] Master 127.10.4.125:34487 requested a full tablet report, sending...
I20260924 00:20:51.434715 10701 heartbeater.cc:347] Connected to a master server at 127.10.4.124:35345
I20260924 00:20:51.435039 10701 heartbeater.cc:464] Registering TS with master...
I20260924 00:20:51.435758 10701 heartbeater.cc:511] Master 127.10.4.124:35345 requested a full tablet report, sending...
I20260924 00:20:51.438598 10374 ts_manager.cc:194] Registered new tserver with Master: b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:20:51.439559 10703 heartbeater.cc:347] Connected to a master server at 127.10.4.126:34479
I20260924 00:20:51.439949 10703 heartbeater.cc:464] Registering TS with master...
I20260924 00:20:51.440703 10703 heartbeater.cc:511] Master 127.10.4.126:34479 requested a full tablet report, sending...
I20260924 00:20:51.442250 10374 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.4.65:55561
I20260924 00:20:51.443545 10299 ts_manager.cc:194] Registered new tserver with Master: b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:20:51.445250 10453 ts_manager.cc:194] Registered new tserver with Master: b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:20:51.520924 10712 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b5aa3debe1cd45e19a6bbd0070913ac3)
I20260924 00:20:51.521435 10712 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } }
I20260924 00:20:51.523254 10712 leader_election.cc:288] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b5aa3debe1cd45e19a6bbd0070913ac3 (127.10.4.125:34487), f8bd0b3184f84f5f8da10c9c21472f92 (127.10.4.124:35345)
I20260924 00:20:51.525017 10384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1d68651913494ce8afee3c136b2da9cc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" is_pre_election: true
I20260924 00:20:51.525372 10463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1d68651913494ce8afee3c136b2da9cc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" is_pre_election: true
I20260924 00:20:51.527077 10286 leader_election.cc:302] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1d68651913494ce8afee3c136b2da9cc; no voters: b5aa3debe1cd45e19a6bbd0070913ac3, f8bd0b3184f84f5f8da10c9c21472f92
I20260924 00:20:51.527782 10712 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260924 00:20:51.727319 10709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:20:51.727815 10709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:20:51.728361 10709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:20:51.759125 10709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:20:51.760090 10709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.4.66
I20260924 00:20:51.796063 10709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.4.84:38335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.4.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.4.66
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.10.4.126:34479,127.10.4.125:34487,127.10.4.124:35345
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.4.66
--log_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:32:10 UTC on ccycloud.quasar-wfxeao.root.comops.site
build id 4668917
TSAN enabled
I20260924 00:20:51.797408 10709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:20:51.799022 10709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:20:51.817611 10719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:20:51.945852 10724 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b5aa3debe1cd45e19a6bbd0070913ac3)
I20260924 00:20:51.946380 10724 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } }
I20260924 00:20:51.948807 10724 leader_election.cc:288] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1d68651913494ce8afee3c136b2da9cc (127.10.4.126:34479), b5aa3debe1cd45e19a6bbd0070913ac3 (127.10.4.125:34487)
I20260924 00:20:51.949059 10309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1d68651913494ce8afee3c136b2da9cc" is_pre_election: true
I20260924 00:20:51.950208 10702 heartbeater.cc:503] Master 127.10.4.125:34487 was elected leader, sending a full tablet report...
I20260924 00:20:51.950627 10384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" is_pre_election: true
I20260924 00:20:51.952085 10440 leader_election.cc:302] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f8bd0b3184f84f5f8da10c9c21472f92; no voters: 1d68651913494ce8afee3c136b2da9cc, b5aa3debe1cd45e19a6bbd0070913ac3
I20260924 00:20:51.953151 10724 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 00:20:52.109740 10725 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b5aa3debe1cd45e19a6bbd0070913ac3)
I20260924 00:20:52.110137 10725 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } }
I20260924 00:20:52.111596 10725 leader_election.cc:288] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b5aa3debe1cd45e19a6bbd0070913ac3 (127.10.4.125:34487), f8bd0b3184f84f5f8da10c9c21472f92 (127.10.4.124:35345)
I20260924 00:20:52.112534 10384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1d68651913494ce8afee3c136b2da9cc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" is_pre_election: true
I20260924 00:20:52.112746 10463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1d68651913494ce8afee3c136b2da9cc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" is_pre_election: true
I20260924 00:20:52.114238 10286 leader_election.cc:302] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1d68651913494ce8afee3c136b2da9cc; no voters: b5aa3debe1cd45e19a6bbd0070913ac3, f8bd0b3184f84f5f8da10c9c21472f92
I20260924 00:20:52.114820 10725 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 00:20:52.621531 10726 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b5aa3debe1cd45e19a6bbd0070913ac3)
I20260924 00:20:52.621899 10726 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } }
I20260924 00:20:52.623330 10726 leader_election.cc:288] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b5aa3debe1cd45e19a6bbd0070913ac3 (127.10.4.125:34487), f8bd0b3184f84f5f8da10c9c21472f92 (127.10.4.124:35345)
I20260924 00:20:52.624444 10384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1d68651913494ce8afee3c136b2da9cc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" is_pre_election: true
I20260924 00:20:52.625324 10463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1d68651913494ce8afee3c136b2da9cc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" is_pre_election: true
I20260924 00:20:52.626858 10286 leader_election.cc:302] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1d68651913494ce8afee3c136b2da9cc; no voters: b5aa3debe1cd45e19a6bbd0070913ac3, f8bd0b3184f84f5f8da10c9c21472f92
I20260924 00:20:52.627647 10726 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260924 00:20:51.817723 10718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:20:51.818122 10721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:20:51.820050 10709 server_base.cc:1034] running on GCE node
I20260924 00:20:52.925567 10709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:20:52.928158 10709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:20:52.929508 10709 hybrid_clock.cc:648] HybridClock initialized: now 1790209252929473 us; error 45 us; skew 500 ppm
I20260924 00:20:52.930279 10709 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 00:20:52.936805 10709 webserver.cc:460] Webserver started at http://127.10.4.66:45227/ using document root <none> and password file <none>
I20260924 00:20:52.937718 10709 fs_manager.cc:359] Metadata directory not provided
I20260924 00:20:52.937935 10709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:20:52.938359 10709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:20:52.944151 10709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/data/instance:
uuid: "48fbd603abe146c585d15196e1b5be08"
format_stamp: "Formatted at 2026-09-24 00:20:52 on dist-test-slave-g3nd"
I20260924 00:20:52.945304 10709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/instance:
uuid: "48fbd603abe146c585d15196e1b5be08"
format_stamp: "Formatted at 2026-09-24 00:20:52 on dist-test-slave-g3nd"
I20260924 00:20:52.953066 10709 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260924 00:20:52.959038 10731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:20:52.962201 10709 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.006s sys 0.000s
I20260924 00:20:52.962697 10709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/data,/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal
uuid: "48fbd603abe146c585d15196e1b5be08"
format_stamp: "Formatted at 2026-09-24 00:20:52 on dist-test-slave-g3nd"
I20260924 00:20:52.963116 10709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:20:53.003480 10709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:20:53.004937 10709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:20:53.005442 10709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:20:53.008315 10709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:20:53.012555 10709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:20:53.012893 10709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:20:53.013221 10709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:20:53.013425 10709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:20:53.158208 10709 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.66:35359
I20260924 00:20:53.158313 10844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.4.66:35359 every 8 connection(s)
I20260924 00:20:53.160789 10709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/data/info.pb
I20260924 00:20:53.169086 10257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu as pid 10709
I20260924 00:20:53.169767 10257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/instance
I20260924 00:20:53.181267 10257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu
/tmp/dist-test-taskko7HPF/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.4.67:0
--local_ip_for_outbound_sockets=127.10.4.67
--webserver_interface=127.10.4.67
--webserver_port=0
--tserver_master_addrs=127.10.4.126:34479,127.10.4.125:34487,127.10.4.124:35345
--builtin_ntp_servers=127.10.4.84:38335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260924 00:20:53.204741 10847 heartbeater.cc:347] Connected to a master server at 127.10.4.126:34479
I20260924 00:20:53.205199 10847 heartbeater.cc:464] Registering TS with master...
I20260924 00:20:53.206419 10847 heartbeater.cc:511] Master 127.10.4.126:34479 requested a full tablet report, sending...
I20260924 00:20:53.208025 10845 heartbeater.cc:347] Connected to a master server at 127.10.4.124:35345
I20260924 00:20:53.208360 10845 heartbeater.cc:464] Registering TS with master...
I20260924 00:20:53.209129 10845 heartbeater.cc:511] Master 127.10.4.124:35345 requested a full tablet report, sending...
I20260924 00:20:53.209345 10299 ts_manager.cc:194] Registered new tserver with Master: 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:20:53.211624 10453 ts_manager.cc:194] Registered new tserver with Master: 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:20:53.213364 10846 heartbeater.cc:347] Connected to a master server at 127.10.4.125:34487
I20260924 00:20:53.213716 10846 heartbeater.cc:464] Registering TS with master...
I20260924 00:20:53.214541 10846 heartbeater.cc:511] Master 127.10.4.125:34487 requested a full tablet report, sending...
I20260924 00:20:53.216351 10374 ts_manager.cc:194] Registered new tserver with Master: 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:20:53.217754 10374 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.4.66:51189
W20260924 00:20:53.305877 10697 debug-util.cc:398] Leaking SignalData structure 0x7b08000af060 after lost signal to thread 10558
W20260924 00:20:53.306684 10697 debug-util.cc:398] Leaking SignalData structure 0x7b08000aff40 after lost signal to thread 10700
W20260924 00:20:53.507012 10855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:20:53.507624 10855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:20:53.508286 10855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:20:53.539165 10855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:20:53.540102 10855 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.4.67
I20260924 00:20:53.575191 10855 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.4.84:38335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.4.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.4.67
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.10.4.126:34479,127.10.4.125:34487,127.10.4.124:35345
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.4.67
--log_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:32:10 UTC on ccycloud.quasar-wfxeao.root.comops.site
build id 4668917
TSAN enabled
I20260924 00:20:53.576376 10855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:20:53.577865 10855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:20:53.597574 10863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:20:53.721319 10846 heartbeater.cc:503] Master 127.10.4.125:34487 was elected leader, sending a full tablet report...
I20260924 00:20:53.744073 10868 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b5aa3debe1cd45e19a6bbd0070913ac3)
I20260924 00:20:53.744880 10868 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } }
I20260924 00:20:53.747189 10868 leader_election.cc:288] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b5aa3debe1cd45e19a6bbd0070913ac3 (127.10.4.125:34487), f8bd0b3184f84f5f8da10c9c21472f92 (127.10.4.124:35345)
I20260924 00:20:53.748337 10384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1d68651913494ce8afee3c136b2da9cc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" is_pre_election: true
I20260924 00:20:53.748644 10463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1d68651913494ce8afee3c136b2da9cc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" is_pre_election: true
I20260924 00:20:53.750393 10286 leader_election.cc:302] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1d68651913494ce8afee3c136b2da9cc; no voters: b5aa3debe1cd45e19a6bbd0070913ac3, f8bd0b3184f84f5f8da10c9c21472f92
I20260924 00:20:53.751184 10868 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 00:20:54.330776 10869 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b5aa3debe1cd45e19a6bbd0070913ac3)
I20260924 00:20:54.331399 10869 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } }
I20260924 00:20:54.335836 10309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1d68651913494ce8afee3c136b2da9cc" is_pre_election: true
I20260924 00:20:54.339366 10869 leader_election.cc:288] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1d68651913494ce8afee3c136b2da9cc (127.10.4.126:34479), b5aa3debe1cd45e19a6bbd0070913ac3 (127.10.4.125:34487)
I20260924 00:20:54.339550 10384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" is_pre_election: true
I20260924 00:20:54.341014 10440 leader_election.cc:302] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f8bd0b3184f84f5f8da10c9c21472f92; no voters: 1d68651913494ce8afee3c136b2da9cc, b5aa3debe1cd45e19a6bbd0070913ac3
I20260924 00:20:54.342111 10869 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260924 00:20:53.600518 10862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:20:54.720685 10864 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection time-out
W20260924 00:20:53.597615 10865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:20:54.721272 10855 server_base.cc:1029] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260924 00:20:54.725561 10855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:20:54.728070 10855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:20:54.729482 10855 hybrid_clock.cc:648] HybridClock initialized: now 1790209254729419 us; error 66 us; skew 500 ppm
I20260924 00:20:54.730473 10855 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 00:20:54.737905 10855 webserver.cc:460] Webserver started at http://127.10.4.67:36887/ using document root <none> and password file <none>
I20260924 00:20:54.738799 10855 fs_manager.cc:359] Metadata directory not provided
I20260924 00:20:54.739010 10855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:20:54.739439 10855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:20:54.744118 10855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/data/instance:
uuid: "c8626e300d0644b0908797ad61bd7cef"
format_stamp: "Formatted at 2026-09-24 00:20:54 on dist-test-slave-g3nd"
I20260924 00:20:54.745245 10855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/instance:
uuid: "c8626e300d0644b0908797ad61bd7cef"
format_stamp: "Formatted at 2026-09-24 00:20:54 on dist-test-slave-g3nd"
I20260924 00:20:54.753185 10855 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260924 00:20:54.759701 10874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:20:54.760805 10855 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260924 00:20:54.761158 10855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/data,/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal
uuid: "c8626e300d0644b0908797ad61bd7cef"
format_stamp: "Formatted at 2026-09-24 00:20:54 on dist-test-slave-g3nd"
I20260924 00:20:54.761495 10855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:20:54.796564 10855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:20:54.797972 10855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:20:54.798378 10855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:20:54.801656 10855 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:20:54.805728 10855 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:20:54.805931 10855 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:20:54.806164 10855 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:20:54.806305 10855 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:20:54.954886 10855 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.67:41301
I20260924 00:20:54.955037 10987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.4.67:41301 every 8 connection(s)
I20260924 00:20:54.958869 10855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/data/info.pb
I20260924 00:20:54.965221 10257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu as pid 10855
I20260924 00:20:54.965654 10257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/instance
I20260924 00:20:55.016131 10989 heartbeater.cc:347] Connected to a master server at 127.10.4.125:34487
I20260924 00:20:55.016594 10989 heartbeater.cc:464] Registering TS with master...
I20260924 00:20:55.016776 10988 heartbeater.cc:347] Connected to a master server at 127.10.4.124:35345
I20260924 00:20:55.017102 10988 heartbeater.cc:464] Registering TS with master...
I20260924 00:20:55.017920 10989 heartbeater.cc:511] Master 127.10.4.125:34487 requested a full tablet report, sending...
I20260924 00:20:55.018021 10988 heartbeater.cc:511] Master 127.10.4.124:35345 requested a full tablet report, sending...
I20260924 00:20:55.020741 10374 ts_manager.cc:194] Registered new tserver with Master: c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:20:55.021325 10453 ts_manager.cc:194] Registered new tserver with Master: c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:20:55.022743 10374 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.4.67:49569
I20260924 00:20:55.023394 10992 heartbeater.cc:347] Connected to a master server at 127.10.4.126:34479
I20260924 00:20:55.023686 10992 heartbeater.cc:464] Registering TS with master...
I20260924 00:20:55.024341 10992 heartbeater.cc:511] Master 127.10.4.126:34479 requested a full tablet report, sending...
I20260924 00:20:55.026572 10299 ts_manager.cc:194] Registered new tserver with Master: c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:20:55.033870 10257 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 00:20:55.114126 10257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu with pid 10411
I20260924 00:20:55.087594 10372 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_7dfad02b25e240ee908f76c201c427ba"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:20:55.086570 10373 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_e0de781451524ba49aa6630c8b98e9c5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:20:55.120115 10373 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e0de781451524ba49aa6630c8b98e9c5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260924 00:20:55.120020 10372 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7dfad02b25e240ee908f76c201c427ba in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:55.089005 10371 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_1341ba5c053445f586f0a89f60843926"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:20:55.122327 10371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1341ba5c053445f586f0a89f60843926 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:55.085829 10374 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_5230088306fa4b8bbf8eca2b74af7248"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:20:55.124320 10374 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5230088306fa4b8bbf8eca2b74af7248 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260924 00:20:55.149204 10361 connection.cc:535] client connection to 127.10.4.124:35345 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260924 00:20:55.149682 10361 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107) [suppressed 6 similar messages]
I20260924 00:20:55.156164 10373 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_e0de781451524ba49aa6630c8b98e9c5" } new_table_name: "default.Table_bf986279c4534aa9bf4f06adb96f16ea" modify_external_catalogs: true
I20260924 00:20:55.180785 10372 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_7dfad02b25e240ee908f76c201c427ba" } new_table_name: "default.Table_bce0831129374039a3abc78db8fc9c7a" modify_external_catalogs: true
I20260924 00:20:55.182499 10374 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_5230088306fa4b8bbf8eca2b74af7248" } modify_external_catalogs: true
I20260924 00:20:55.182992 10371 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_1341ba5c053445f586f0a89f60843926" } modify_external_catalogs: true
I20260924 00:20:55.183050 10374 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_5230088306fa4b8bbf8eca2b74af7248" } modify_external_catalogs: true
W20260924 00:20:55.183192 10361 consensus_peers.cc:487] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 -> Peer f8bd0b3184f84f5f8da10c9c21472f92 (127.10.4.124:35345): Couldn't send request to peer f8bd0b3184f84f5f8da10c9c21472f92. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.124:35345: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 00:20:55.183507 10371 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_1341ba5c053445f586f0a89f60843926" } modify_external_catalogs: true
W20260924 00:20:55.205819 10373 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 84a455f7221148de87e2c3a5995afd5d consensus state has no leader
I20260924 00:20:55.229501 10371 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 78cc4da82bef495280cc8f217559b531
I20260924 00:20:55.229370 10374 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet cd8e7d4f384945f0894a8f1e0b4a2635
I20260924 00:20:55.235754 10922 tablet_service.cc:1467] Processing CreateTablet for tablet 84a455f7221148de87e2c3a5995afd5d (DEFAULT_TABLE table=default.Table_e0de781451524ba49aa6630c8b98e9c5 [id=43419db539614db8802a84dff868f6b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:55.237936 10922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84a455f7221148de87e2c3a5995afd5d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:20:55.238057 10372 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 58d19cff0804461d9b4deca952b94386 has no consensus state
W20260924 00:20:55.239578 10361 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 84a455f7221148de87e2c3a5995afd5d (table default.Table_bf986279c4534aa9bf4f06adb96f16ea [id=43419db539614db8802a84dff868f6b9], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 84a455f7221148de87e2c3a5995afd5d consensus state has no leader
I20260924 00:20:55.246086 10636 tablet_service.cc:1467] Processing CreateTablet for tablet 84a455f7221148de87e2c3a5995afd5d (DEFAULT_TABLE table=default.Table_e0de781451524ba49aa6630c8b98e9c5 [id=43419db539614db8802a84dff868f6b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:55.246546 10780 tablet_service.cc:1467] Processing CreateTablet for tablet 84a455f7221148de87e2c3a5995afd5d (DEFAULT_TABLE table=default.Table_e0de781451524ba49aa6630c8b98e9c5 [id=43419db539614db8802a84dff868f6b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:55.248262 10780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84a455f7221148de87e2c3a5995afd5d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:20:55.248595 10845 heartbeater.cc:644] Failed to heartbeat to 127.10.4.124:35345 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.4.124:35345: connect: Connection refused (error 111)
I20260924 00:20:55.248241 10636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84a455f7221148de87e2c3a5995afd5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:55.254622 10779 tablet_service.cc:1467] Processing CreateTablet for tablet 58d19cff0804461d9b4deca952b94386 (DEFAULT_TABLE table=default.Table_bce0831129374039a3abc78db8fc9c7a [id=143547db8d2d4517877751965d3042ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:55.255381 10635 tablet_service.cc:1467] Processing CreateTablet for tablet 58d19cff0804461d9b4deca952b94386 (DEFAULT_TABLE table=default.Table_bce0831129374039a3abc78db8fc9c7a [id=143547db8d2d4517877751965d3042ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:55.255705 10779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58d19cff0804461d9b4deca952b94386. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:55.256414 10635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58d19cff0804461d9b4deca952b94386. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:55.258634 10921 tablet_service.cc:1467] Processing CreateTablet for tablet 58d19cff0804461d9b4deca952b94386 (DEFAULT_TABLE table=default.Table_bce0831129374039a3abc78db8fc9c7a [id=143547db8d2d4517877751965d3042ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:55.259768 10921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58d19cff0804461d9b4deca952b94386. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:55.298442 11039 tablet_bootstrap.cc:492] T 84a455f7221148de87e2c3a5995afd5d P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:20:55.298578 11040 tablet_bootstrap.cc:492] T 84a455f7221148de87e2c3a5995afd5d P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:20:55.299929 11041 tablet_bootstrap.cc:492] T 58d19cff0804461d9b4deca952b94386 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:20:55.303459 11039 tablet_bootstrap.cc:654] T 84a455f7221148de87e2c3a5995afd5d P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:55.305215 11040 tablet_bootstrap.cc:654] T 84a455f7221148de87e2c3a5995afd5d P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:55.305655 11039 log.cc:824] T 84a455f7221148de87e2c3a5995afd5d P c8626e300d0644b0908797ad61bd7cef: Log is configured to *not* fsync() on all Append() calls
I20260924 00:20:55.306458 11041 tablet_bootstrap.cc:654] T 58d19cff0804461d9b4deca952b94386 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:55.307325 11040 log.cc:824] T 84a455f7221148de87e2c3a5995afd5d P 48fbd603abe146c585d15196e1b5be08: Log is configured to *not* fsync() on all Append() calls
I20260924 00:20:55.308524 11041 log.cc:824] T 58d19cff0804461d9b4deca952b94386 P b10a9082a37b46f09d581944914c1670: Log is configured to *not* fsync() on all Append() calls
I20260924 00:20:55.310468 11039 tablet_bootstrap.cc:492] T 84a455f7221148de87e2c3a5995afd5d P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:20:55.310889 11039 ts_tablet_manager.cc:1397] T 84a455f7221148de87e2c3a5995afd5d P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.003s
I20260924 00:20:55.312011 11040 tablet_bootstrap.cc:492] T 84a455f7221148de87e2c3a5995afd5d P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:20:55.312460 11040 ts_tablet_manager.cc:1397] T 84a455f7221148de87e2c3a5995afd5d P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.008s
I20260924 00:20:55.313942 11041 tablet_bootstrap.cc:492] T 58d19cff0804461d9b4deca952b94386 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:20:55.314471 11041 ts_tablet_manager.cc:1397] T 58d19cff0804461d9b4deca952b94386 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.004s
I20260924 00:20:55.339169 10372 master_service.cc:946] ReplaceTablet: received request to replace tablet 84a455f7221148de87e2c3a5995afd5d from {username='slave'} at 127.0.0.1:47022
I20260924 00:20:55.342672 10374 master_service.cc:946] ReplaceTablet: received request to replace tablet 58d19cff0804461d9b4deca952b94386 from {username='slave'} at 127.0.0.1:47022
I20260924 00:20:55.342422 11040 raft_consensus.cc:348] T 84a455f7221148de87e2c3a5995afd5d P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:20:55.343474 11040 raft_consensus.cc:374] T 84a455f7221148de87e2c3a5995afd5d P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:55.343832 11040 raft_consensus.cc:729] T 84a455f7221148de87e2c3a5995afd5d P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:20:55.343922 11041 raft_consensus.cc:348] T 58d19cff0804461d9b4deca952b94386 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:55.344782 11041 raft_consensus.cc:374] T 58d19cff0804461d9b4deca952b94386 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:55.345121 11041 raft_consensus.cc:729] T 58d19cff0804461d9b4deca952b94386 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:20:55.344832 11040 consensus_queue.cc:260] T 84a455f7221148de87e2c3a5995afd5d P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:20:55.345976 11041 consensus_queue.cc:260] T 58d19cff0804461d9b4deca952b94386 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:55.357117 11041 ts_tablet_manager.cc:1428] T 58d19cff0804461d9b4deca952b94386 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.042s user 0.029s sys 0.011s
I20260924 00:20:55.356587 10371 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_78b48660f5674e7d90595b7ef0182966"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:20:55.358507 10371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_78b48660f5674e7d90595b7ef0182966 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:55.365808 11041 tablet_bootstrap.cc:492] T 84a455f7221148de87e2c3a5995afd5d P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:20:55.366042 11040 ts_tablet_manager.cc:1428] T 84a455f7221148de87e2c3a5995afd5d P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.053s user 0.037s sys 0.003s
I20260924 00:20:55.369107 11040 tablet_bootstrap.cc:492] T 58d19cff0804461d9b4deca952b94386 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:20:55.371944 10373 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_9f1044a5b2064646bb2697c073aa755a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:20:55.373557 10373 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9f1044a5b2064646bb2697c073aa755a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:55.374917 11040 tablet_bootstrap.cc:654] T 58d19cff0804461d9b4deca952b94386 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:55.375773 10370 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_a7c75883d8ef40f3bdf2ca5a734cbe6f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:20:55.377491 10370 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a7c75883d8ef40f3bdf2ca5a734cbe6f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:55.382897 11040 tablet_bootstrap.cc:492] T 58d19cff0804461d9b4deca952b94386 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:20:55.383368 11040 ts_tablet_manager.cc:1397] T 58d19cff0804461d9b4deca952b94386 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.003s
I20260924 00:20:55.372835 10374 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 58d19cff0804461d9b4deca952b94386
I20260924 00:20:55.383917 10368 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_2d9dbcf42731420199dabcdd3f6df616"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:20:55.385809 10368 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2d9dbcf42731420199dabcdd3f6df616 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:55.385910 11040 raft_consensus.cc:348] T 58d19cff0804461d9b4deca952b94386 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:55.386281 10635 tablet_service.cc:1514] Processing DeleteTablet for tablet 58d19cff0804461d9b4deca952b94386 with delete_type TABLET_DATA_DELETED (replaced by tablet 61c5ea5aefa3448ca62abe8bb6eec2e8) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:55.386557 11040 raft_consensus.cc:374] T 58d19cff0804461d9b4deca952b94386 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:55.386817 11040 raft_consensus.cc:729] T 58d19cff0804461d9b4deca952b94386 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:20:55.387530 11040 consensus_queue.cc:260] T 58d19cff0804461d9b4deca952b94386 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:55.390064 11040 ts_tablet_manager.cc:1428] T 58d19cff0804461d9b4deca952b94386 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260924 00:20:55.391072 10374 catalog_manager.cc:6565] ReplaceTablet: tablet 58d19cff0804461d9b4deca952b94386 deleted and replaced by tablet 61c5ea5aefa3448ca62abe8bb6eec2e8
I20260924 00:20:55.392210 10780 tablet_service.cc:1514] Processing DeleteTablet for tablet 58d19cff0804461d9b4deca952b94386 with delete_type TABLET_DATA_DELETED (replaced by tablet 61c5ea5aefa3448ca62abe8bb6eec2e8) from {username='slave'} at 127.0.0.1:35996
I20260924 00:20:55.392944 10922 tablet_service.cc:1514] Processing DeleteTablet for tablet 58d19cff0804461d9b4deca952b94386 with delete_type TABLET_DATA_DELETED (replaced by tablet 61c5ea5aefa3448ca62abe8bb6eec2e8) from {username='slave'} at 127.0.0.1:35358
W20260924 00:20:55.403110 10706 tablet.cc:2369] T 58d19cff0804461d9b4deca952b94386 P b10a9082a37b46f09d581944914c1670: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:20:55.413785 11050 tablet_replica.cc:330] T 58d19cff0804461d9b4deca952b94386 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:20:55.414917 11041 tablet_bootstrap.cc:654] T 84a455f7221148de87e2c3a5995afd5d P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:55.415997 11050 raft_consensus.cc:2229] T 58d19cff0804461d9b4deca952b94386 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:55.420712 11049 tablet_replica.cc:330] T 58d19cff0804461d9b4deca952b94386 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:20:55.421743 11050 raft_consensus.cc:2258] T 58d19cff0804461d9b4deca952b94386 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:55.422537 11049 raft_consensus.cc:2229] T 58d19cff0804461d9b4deca952b94386 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:55.423058 11049 raft_consensus.cc:2258] T 58d19cff0804461d9b4deca952b94386 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:55.423954 11050 ts_tablet_manager.cc:1905] T 58d19cff0804461d9b4deca952b94386 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:55.427762 11049 ts_tablet_manager.cc:1905] T 58d19cff0804461d9b4deca952b94386 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:55.409771 10371 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_78b48660f5674e7d90595b7ef0182966" } new_table_name: "default.Table_6f33bd5289bf46dcb45c4abf247b61c1" modify_external_catalogs: true
W20260924 00:20:55.421947 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 58d19cff0804461d9b4deca952b94386 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 58d19cff0804461d9b4deca952b94386 already in progress: creating tablet
W20260924 00:20:55.434554 10849 tablet.cc:2369] T 84a455f7221148de87e2c3a5995afd5d P 48fbd603abe146c585d15196e1b5be08: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:20:55.438202 11041 tablet_bootstrap.cc:492] T 84a455f7221148de87e2c3a5995afd5d P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:20:55.438647 11041 ts_tablet_manager.cc:1397] T 84a455f7221148de87e2c3a5995afd5d P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.074s user 0.010s sys 0.002s
I20260924 00:20:55.444131 10370 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_bce0831129374039a3abc78db8fc9c7a" } new_table_name: "default.Table_d4b54eed2b8a4b3994b2635e01c5c273" modify_external_catalogs: true
I20260924 00:20:55.445175 11049 ts_tablet_manager.cc:1918] T 58d19cff0804461d9b4deca952b94386 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:55.445609 11049 log.cc:1196] T 58d19cff0804461d9b4deca952b94386 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/58d19cff0804461d9b4deca952b94386
I20260924 00:20:55.446563 11049 ts_tablet_manager.cc:1939] T 58d19cff0804461d9b4deca952b94386 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:20:55.446692 10374 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_9f1044a5b2064646bb2697c073aa755a" } modify_external_catalogs: true
I20260924 00:20:55.447288 10374 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_9f1044a5b2064646bb2697c073aa755a" } modify_external_catalogs: true
I20260924 00:20:55.447914 11041 raft_consensus.cc:348] T 84a455f7221148de87e2c3a5995afd5d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:20:55.448652 11041 raft_consensus.cc:374] T 84a455f7221148de87e2c3a5995afd5d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:55.448994 11041 raft_consensus.cc:729] T 84a455f7221148de87e2c3a5995afd5d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:20:55.449673 11041 consensus_queue.cc:260] T 84a455f7221148de87e2c3a5995afd5d P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:20:55.451272 11041 ts_tablet_manager.cc:1428] T 84a455f7221148de87e2c3a5995afd5d P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.012s user 0.003s sys 0.000s
I20260924 00:20:55.454046 11050 ts_tablet_manager.cc:1918] T 58d19cff0804461d9b4deca952b94386 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:55.449682 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 58d19cff0804461d9b4deca952b94386 (table default.Table_bce0831129374039a3abc78db8fc9c7a [id=143547db8d2d4517877751965d3042ce]) successfully deleted
I20260924 00:20:55.454526 11050 log.cc:1196] T 58d19cff0804461d9b4deca952b94386 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/58d19cff0804461d9b4deca952b94386
I20260924 00:20:55.455256 11050 ts_tablet_manager.cc:1939] T 58d19cff0804461d9b4deca952b94386 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:20:55.454180 10373 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_bf986279c4534aa9bf4f06adb96f16ea" } modify_external_catalogs: true
I20260924 00:20:55.456233 10373 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_bf986279c4534aa9bf4f06adb96f16ea" } modify_external_catalogs: true
I20260924 00:20:55.458403 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 58d19cff0804461d9b4deca952b94386 (table default.Table_bce0831129374039a3abc78db8fc9c7a [id=143547db8d2d4517877751965d3042ce]) successfully deleted
W20260924 00:20:55.464543 10701 heartbeater.cc:644] Failed to heartbeat to 127.10.4.124:35345 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.4.124:35345: connect: Connection refused (error 111)
I20260924 00:20:55.466389 10372 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 84a455f7221148de87e2c3a5995afd5d
I20260924 00:20:55.467986 10922 tablet_service.cc:1514] Processing DeleteTablet for tablet 84a455f7221148de87e2c3a5995afd5d with delete_type TABLET_DATA_DELETED (replaced by tablet 8fc17bdba09b40b7a3280c64121ff508) from {username='slave'} at 127.0.0.1:35358
W20260924 00:20:55.469585 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 84a455f7221148de87e2c3a5995afd5d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 84a455f7221148de87e2c3a5995afd5d already in progress: creating tablet
I20260924 00:20:55.471518 10780 tablet_service.cc:1514] Processing DeleteTablet for tablet 84a455f7221148de87e2c3a5995afd5d with delete_type TABLET_DATA_DELETED (replaced by tablet 8fc17bdba09b40b7a3280c64121ff508) from {username='slave'} at 127.0.0.1:35996
I20260924 00:20:55.471756 10372 catalog_manager.cc:6565] ReplaceTablet: tablet 84a455f7221148de87e2c3a5995afd5d deleted and replaced by tablet 8fc17bdba09b40b7a3280c64121ff508
I20260924 00:20:55.472345 11050 tablet_replica.cc:330] T 84a455f7221148de87e2c3a5995afd5d P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:20:55.473050 11050 raft_consensus.cc:2229] T 84a455f7221148de87e2c3a5995afd5d P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:55.473551 11050 raft_consensus.cc:2258] T 84a455f7221148de87e2c3a5995afd5d P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260924 00:20:55.475178 10371 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1e5beeac2a1d4cb18b5d3d2408fd574b has no consensus state
I20260924 00:20:55.475584 11050 ts_tablet_manager.cc:1905] T 84a455f7221148de87e2c3a5995afd5d P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:55.477113 10635 tablet_service.cc:1514] Processing DeleteTablet for tablet 84a455f7221148de87e2c3a5995afd5d with delete_type TABLET_DATA_DELETED (replaced by tablet 8fc17bdba09b40b7a3280c64121ff508) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:55.477383 11039 raft_consensus.cc:348] T 84a455f7221148de87e2c3a5995afd5d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:20:55.478199 11049 tablet_replica.cc:330] T 84a455f7221148de87e2c3a5995afd5d P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:20:55.478309 11039 raft_consensus.cc:374] T 84a455f7221148de87e2c3a5995afd5d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:55.478624 11039 raft_consensus.cc:729] T 84a455f7221148de87e2c3a5995afd5d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:20:55.478824 11049 raft_consensus.cc:2229] T 84a455f7221148de87e2c3a5995afd5d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:55.479277 11049 raft_consensus.cc:2258] T 84a455f7221148de87e2c3a5995afd5d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:55.479632 11039 consensus_queue.cc:260] T 84a455f7221148de87e2c3a5995afd5d P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:20:55.481382 11049 ts_tablet_manager.cc:1905] T 84a455f7221148de87e2c3a5995afd5d P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260924 00:20:55.491950 10370 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 61c5ea5aefa3448ca62abe8bb6eec2e8 has no consensus state
I20260924 00:20:55.495692 11049 ts_tablet_manager.cc:1918] T 84a455f7221148de87e2c3a5995afd5d P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:55.496155 11049 log.cc:1196] T 84a455f7221148de87e2c3a5995afd5d P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/84a455f7221148de87e2c3a5995afd5d
I20260924 00:20:55.496747 10635 tablet_service.cc:1514] Processing DeleteTablet for tablet 58d19cff0804461d9b4deca952b94386 with delete_type TABLET_DATA_DELETED (replaced by tablet 61c5ea5aefa3448ca62abe8bb6eec2e8) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:55.497749 11049 ts_tablet_manager.cc:1939] T 84a455f7221148de87e2c3a5995afd5d P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:20:55.500252 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 84a455f7221148de87e2c3a5995afd5d (table default.Table_bf986279c4534aa9bf4f06adb96f16ea [id=43419db539614db8802a84dff868f6b9]) successfully deleted
W20260924 00:20:55.502229 10359 catalog_manager.cc:4808] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 58d19cff0804461d9b4deca952b94386 because the tablet was not found. No further retry: Not found: Tablet not found: 58d19cff0804461d9b4deca952b94386
I20260924 00:20:55.504062 11050 ts_tablet_manager.cc:1918] T 84a455f7221148de87e2c3a5995afd5d P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:55.504238 11039 ts_tablet_manager.cc:1428] T 84a455f7221148de87e2c3a5995afd5d P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.193s user 0.038s sys 0.015s
I20260924 00:20:55.505004 10989 heartbeater.cc:503] Master 127.10.4.125:34487 was elected leader, sending a full tablet report...
I20260924 00:20:55.506045 11050 log.cc:1196] T 84a455f7221148de87e2c3a5995afd5d P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/84a455f7221148de87e2c3a5995afd5d
I20260924 00:20:55.506721 11050 ts_tablet_manager.cc:1939] T 84a455f7221148de87e2c3a5995afd5d P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
W20260924 00:20:55.509498 10988 heartbeater.cc:644] Failed to heartbeat to 127.10.4.124:35345 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.4.124:35345: connect: Connection refused (error 111)
I20260924 00:20:55.511754 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 84a455f7221148de87e2c3a5995afd5d (table default.Table_bf986279c4534aa9bf4f06adb96f16ea [id=43419db539614db8802a84dff868f6b9]) successfully deleted
I20260924 00:20:55.511960 11039 tablet_bootstrap.cc:492] T 58d19cff0804461d9b4deca952b94386 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:20:55.517369 11039 tablet_bootstrap.cc:654] T 58d19cff0804461d9b4deca952b94386 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:55.522794 11039 tablet_bootstrap.cc:492] T 58d19cff0804461d9b4deca952b94386 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:20:55.523109 11039 ts_tablet_manager.cc:1397] T 58d19cff0804461d9b4deca952b94386 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260924 00:20:55.525373 11039 raft_consensus.cc:348] T 58d19cff0804461d9b4deca952b94386 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:55.526005 11039 raft_consensus.cc:374] T 58d19cff0804461d9b4deca952b94386 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:55.526378 11039 raft_consensus.cc:729] T 58d19cff0804461d9b4deca952b94386 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:20:55.527109 11039 consensus_queue.cc:260] T 58d19cff0804461d9b4deca952b94386 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:55.529721 11039 ts_tablet_manager.cc:1428] T 58d19cff0804461d9b4deca952b94386 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260924 00:20:55.532819 10635 tablet_service.cc:1467] Processing CreateTablet for tablet 61c5ea5aefa3448ca62abe8bb6eec2e8 (DEFAULT_TABLE table=default.Table_d4b54eed2b8a4b3994b2635e01c5c273 [id=143547db8d2d4517877751965d3042ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:55.533872 10635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61c5ea5aefa3448ca62abe8bb6eec2e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:55.535125 10780 tablet_service.cc:1467] Processing CreateTablet for tablet 61c5ea5aefa3448ca62abe8bb6eec2e8 (DEFAULT_TABLE table=default.Table_d4b54eed2b8a4b3994b2635e01c5c273 [id=143547db8d2d4517877751965d3042ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:55.536100 10780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61c5ea5aefa3448ca62abe8bb6eec2e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:55.537228 10922 tablet_service.cc:1467] Processing CreateTablet for tablet 61c5ea5aefa3448ca62abe8bb6eec2e8 (DEFAULT_TABLE table=default.Table_d4b54eed2b8a4b3994b2635e01c5c273 [id=143547db8d2d4517877751965d3042ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:55.538187 10922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61c5ea5aefa3448ca62abe8bb6eec2e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:55.547060 10374 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 2bf53014d60d4e70a435e4878b717bc6
I20260924 00:20:55.550014 11040 tablet_bootstrap.cc:492] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:20:55.551532 11041 tablet_bootstrap.cc:492] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:20:55.555693 11040 tablet_bootstrap.cc:654] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:55.557037 11041 tablet_bootstrap.cc:654] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:55.569269 10373 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 8fc17bdba09b40b7a3280c64121ff508
I20260924 00:20:55.574747 11040 tablet_bootstrap.cc:492] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:20:55.575109 10780 tablet_service.cc:1514] Processing DeleteTablet for tablet 84a455f7221148de87e2c3a5995afd5d with delete_type TABLET_DATA_DELETED (replaced by tablet 8fc17bdba09b40b7a3280c64121ff508) from {username='slave'} at 127.0.0.1:35996
W20260924 00:20:55.577582 10361 catalog_manager.cc:4808] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 84a455f7221148de87e2c3a5995afd5d because the tablet was not found. No further retry: Not found: Tablet not found: 84a455f7221148de87e2c3a5995afd5d
I20260924 00:20:55.577733 10635 tablet_service.cc:1467] Processing CreateTablet for tablet 971e0913f7454a4a9ec25ac3891d2b1e (DEFAULT_TABLE table=default.Table_2d9dbcf42731420199dabcdd3f6df616 [id=e6c357c8899f430b95c9c55084999d6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:55.578727 10635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 971e0913f7454a4a9ec25ac3891d2b1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:55.579934 10780 tablet_service.cc:1467] Processing CreateTablet for tablet 971e0913f7454a4a9ec25ac3891d2b1e (DEFAULT_TABLE table=default.Table_2d9dbcf42731420199dabcdd3f6df616 [id=e6c357c8899f430b95c9c55084999d6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:55.581084 10780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 971e0913f7454a4a9ec25ac3891d2b1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:55.582676 10636 tablet_service.cc:1467] Processing CreateTablet for tablet e1864ca7a2e44611b6bb4e87f04dc61f (DEFAULT_TABLE table=default.Table_a7c75883d8ef40f3bdf2ca5a734cbe6f [id=9d8bc2174e884f97b364cc1d2e5af3b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:55.583666 10636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1864ca7a2e44611b6bb4e87f04dc61f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:55.583680 10634 tablet_service.cc:1467] Processing CreateTablet for tablet 1e5beeac2a1d4cb18b5d3d2408fd574b (DEFAULT_TABLE table=default.Table_6f33bd5289bf46dcb45c4abf247b61c1 [id=30a3f7c8fb9c46cf8b883eedae25d938]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:55.584764 10634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e5beeac2a1d4cb18b5d3d2408fd574b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:55.585652 10921 tablet_service.cc:1467] Processing CreateTablet for tablet 971e0913f7454a4a9ec25ac3891d2b1e (DEFAULT_TABLE table=default.Table_2d9dbcf42731420199dabcdd3f6df616 [id=e6c357c8899f430b95c9c55084999d6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:55.586642 10921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 971e0913f7454a4a9ec25ac3891d2b1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:55.588035 10633 tablet_service.cc:1467] Processing CreateTablet for tablet 8fc17bdba09b40b7a3280c64121ff508 (DEFAULT_TABLE table=default.Table_bf986279c4534aa9bf4f06adb96f16ea [id=43419db539614db8802a84dff868f6b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:55.589151 10633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fc17bdba09b40b7a3280c64121ff508. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:55.589727 10920 tablet_service.cc:1467] Processing CreateTablet for tablet e1864ca7a2e44611b6bb4e87f04dc61f (DEFAULT_TABLE table=default.Table_a7c75883d8ef40f3bdf2ca5a734cbe6f [id=9d8bc2174e884f97b364cc1d2e5af3b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:55.590700 10920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1864ca7a2e44611b6bb4e87f04dc61f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:55.590835 10777 tablet_service.cc:1467] Processing CreateTablet for tablet 8fc17bdba09b40b7a3280c64121ff508 (DEFAULT_TABLE table=default.Table_bf986279c4534aa9bf4f06adb96f16ea [id=43419db539614db8802a84dff868f6b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:55.591859 10777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fc17bdba09b40b7a3280c64121ff508. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:55.593317 11039 tablet_bootstrap.cc:492] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:20:55.596616 10778 tablet_service.cc:1467] Processing CreateTablet for tablet 1e5beeac2a1d4cb18b5d3d2408fd574b (DEFAULT_TABLE table=default.Table_6f33bd5289bf46dcb45c4abf247b61c1 [id=30a3f7c8fb9c46cf8b883eedae25d938]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:55.597733 10778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e5beeac2a1d4cb18b5d3d2408fd574b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:55.597666 10779 tablet_service.cc:1467] Processing CreateTablet for tablet e1864ca7a2e44611b6bb4e87f04dc61f (DEFAULT_TABLE table=default.Table_a7c75883d8ef40f3bdf2ca5a734cbe6f [id=9d8bc2174e884f97b364cc1d2e5af3b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:55.598631 10779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1864ca7a2e44611b6bb4e87f04dc61f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:55.599009 11056 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } } }
I20260924 00:20:55.599814 11056 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [sys.catalog]: This master's current role is: LEADER
I20260924 00:20:55.601182 10632 tablet_service.cc:1514] Processing DeleteTablet for tablet 84a455f7221148de87e2c3a5995afd5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:55 UTC) from {username='slave'} at 127.0.0.1:46896
W20260924 00:20:55.602643 10359 catalog_manager.cc:4808] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 84a455f7221148de87e2c3a5995afd5d because the tablet was not found. No further retry: Not found: Tablet not found: 84a455f7221148de87e2c3a5995afd5d
I20260924 00:20:55.575126 11040 ts_tablet_manager.cc:1397] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.025s user 0.010s sys 0.000s
I20260924 00:20:55.607015 10919 tablet_service.cc:1514] Processing DeleteTablet for tablet 84a455f7221148de87e2c3a5995afd5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:55 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:20:55.607358 10918 tablet_service.cc:1514] Processing DeleteTablet for tablet 58d19cff0804461d9b4deca952b94386 with delete_type TABLET_DATA_DELETED (replaced by tablet 61c5ea5aefa3448ca62abe8bb6eec2e8) from {username='slave'} at 127.0.0.1:35358
I20260924 00:20:55.608384 10917 tablet_service.cc:1467] Processing CreateTablet for tablet 8fc17bdba09b40b7a3280c64121ff508 (DEFAULT_TABLE table=default.Table_bf986279c4534aa9bf4f06adb96f16ea [id=43419db539614db8802a84dff868f6b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:55.609419 10917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fc17bdba09b40b7a3280c64121ff508. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:55.610105 10922 tablet_service.cc:1467] Processing CreateTablet for tablet 1e5beeac2a1d4cb18b5d3d2408fd574b (DEFAULT_TABLE table=default.Table_6f33bd5289bf46dcb45c4abf247b61c1 [id=30a3f7c8fb9c46cf8b883eedae25d938]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:55.611076 10922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e5beeac2a1d4cb18b5d3d2408fd574b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:55.613384 11051 tablet_replica.cc:330] T 84a455f7221148de87e2c3a5995afd5d P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:20:55.614197 11051 raft_consensus.cc:2229] T 84a455f7221148de87e2c3a5995afd5d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:55.614709 11051 raft_consensus.cc:2258] T 84a455f7221148de87e2c3a5995afd5d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:55.618484 11051 ts_tablet_manager.cc:1905] T 84a455f7221148de87e2c3a5995afd5d P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:55.623082 10369 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_0a635e42c38a4b84a1ac9d8838333c9a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:20:55.624965 10369 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0a635e42c38a4b84a1ac9d8838333c9a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:55.626194 10370 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_05ea79d978c74171980cf4ffdc8dd08e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:20:55.628077 10370 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_05ea79d978c74171980cf4ffdc8dd08e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:55.634681 11039 tablet_bootstrap.cc:654] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:55.639869 10367 master_service.cc:946] ReplaceTablet: received request to replace tablet e1864ca7a2e44611b6bb4e87f04dc61f from {username='slave'} at 127.0.0.1:47022
I20260924 00:20:55.648746 10373 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_3452ff0ba69a4ac6a2cdfb298648638b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:20:55.650374 10373 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3452ff0ba69a4ac6a2cdfb298648638b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:55.669970 10367 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet e1864ca7a2e44611b6bb4e87f04dc61f
I20260924 00:20:55.673007 10367 catalog_manager.cc:6565] ReplaceTablet: tablet e1864ca7a2e44611b6bb4e87f04dc61f deleted and replaced by tablet 720e1a36bcfb4260b3689d99a7358205
I20260924 00:20:55.673126 10918 tablet_service.cc:1514] Processing DeleteTablet for tablet e1864ca7a2e44611b6bb4e87f04dc61f with delete_type TABLET_DATA_DELETED (replaced by tablet 720e1a36bcfb4260b3689d99a7358205) from {username='slave'} at 127.0.0.1:35358
I20260924 00:20:55.675375 11041 tablet_bootstrap.cc:492] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:20:55.677023 10778 tablet_service.cc:1514] Processing DeleteTablet for tablet e1864ca7a2e44611b6bb4e87f04dc61f with delete_type TABLET_DATA_DELETED (replaced by tablet 720e1a36bcfb4260b3689d99a7358205) from {username='slave'} at 127.0.0.1:35996
I20260924 00:20:55.675885 11041 ts_tablet_manager.cc:1397] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.125s user 0.016s sys 0.011s
I20260924 00:20:55.681665 10374 master_service.cc:946] ReplaceTablet: received request to replace tablet 61c5ea5aefa3448ca62abe8bb6eec2e8 from {username='slave'} at 127.0.0.1:47022
W20260924 00:20:55.684875 10361 catalog_manager.cc:4808] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet e1864ca7a2e44611b6bb4e87f04dc61f because the tablet was not found. No further retry: Not found: Tablet not found: e1864ca7a2e44611b6bb4e87f04dc61f
I20260924 00:20:55.687399 11040 raft_consensus.cc:348] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:20:55.688185 11040 raft_consensus.cc:374] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:55.688493 11040 raft_consensus.cc:729] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:20:55.690888 10369 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_a7c75883d8ef40f3bdf2ca5a734cbe6f" } new_table_name: "default.Table_b1ae3ebb4b3e4c43a1a684b765cfdd38" modify_external_catalogs: true
I20260924 00:20:55.694814 11039 tablet_bootstrap.cc:492] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:20:55.694711 10367 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_5f8c295237ab4e0ea184a03a9fa458a2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:20:55.695374 11039 ts_tablet_manager.cc:1397] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.102s user 0.010s sys 0.000s
I20260924 00:20:55.698081 11039 raft_consensus.cc:348] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:20:55.698884 11039 raft_consensus.cc:374] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:55.699211 11039 raft_consensus.cc:729] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:20:55.698874 10373 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_3452ff0ba69a4ac6a2cdfb298648638b" } new_table_name: "default.Table_b1726c09065e4fbaa2e6799b3ebdbc74" modify_external_catalogs: true
I20260924 00:20:55.699996 11039 consensus_queue.cc:260] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:20:55.702780 11041 raft_consensus.cc:348] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:20:55.703514 11041 raft_consensus.cc:374] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:55.703815 11041 raft_consensus.cc:729] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:20:55.704547 11041 consensus_queue.cc:260] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:20:55.705717 10636 tablet_service.cc:1514] Processing DeleteTablet for tablet e1864ca7a2e44611b6bb4e87f04dc61f with delete_type TABLET_DATA_DELETED (replaced by tablet 720e1a36bcfb4260b3689d99a7358205) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:55.707376 11041 ts_tablet_manager.cc:1428] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.029s user 0.004s sys 0.000s
I20260924 00:20:55.708245 11041 tablet_bootstrap.cc:492] T 1e5beeac2a1d4cb18b5d3d2408fd574b P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
W20260924 00:20:55.711776 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet e1864ca7a2e44611b6bb4e87f04dc61f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e1864ca7a2e44611b6bb4e87f04dc61f already in progress: creating tablet
I20260924 00:20:55.713598 11041 tablet_bootstrap.cc:654] T 1e5beeac2a1d4cb18b5d3d2408fd574b P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:55.689260 11040 consensus_queue.cc:260] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:20:55.728138 10918 tablet_service.cc:1467] Processing CreateTablet for tablet ad256de63e0948928a51d9231152ac2d (DEFAULT_TABLE table=default.Table_0a635e42c38a4b84a1ac9d8838333c9a [id=f6322db8cfe84fcfbaa47bb7a17e492e]), partition=RANGE (key) PARTITION UNBOUNDED
W20260924 00:20:55.728615 10367 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5f8c295237ab4e0ea184a03a9fa458a2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:55.729194 10918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad256de63e0948928a51d9231152ac2d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:20:55.729972 10993 tablet.cc:2369] T 58d19cff0804461d9b4deca952b94386 P c8626e300d0644b0908797ad61bd7cef: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:20:55.731997 11040 ts_tablet_manager.cc:1428] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.079s user 0.001s sys 0.005s
I20260924 00:20:55.731791 10633 tablet_service.cc:1467] Processing CreateTablet for tablet ad256de63e0948928a51d9231152ac2d (DEFAULT_TABLE table=default.Table_0a635e42c38a4b84a1ac9d8838333c9a [id=f6322db8cfe84fcfbaa47bb7a17e492e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:55.732776 10633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad256de63e0948928a51d9231152ac2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:55.737651 10636 tablet_service.cc:1467] Processing CreateTablet for tablet 746fb7b4571f428a80495f6e9c262a7f (DEFAULT_TABLE table=default.Table_05ea79d978c74171980cf4ffdc8dd08e [id=96e09747aa014d5bb96e5d991b38c039]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:55.738644 10636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 746fb7b4571f428a80495f6e9c262a7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:55.741854 11040 tablet_bootstrap.cc:492] T 1e5beeac2a1d4cb18b5d3d2408fd574b P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:20:55.748332 11051 ts_tablet_manager.cc:1918] T 84a455f7221148de87e2c3a5995afd5d P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:55.748185 11040 tablet_bootstrap.cc:654] T 1e5beeac2a1d4cb18b5d3d2408fd574b P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
W20260924 00:20:55.748898 10373 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a1a79421de4049aea258261ff9bd00b4 has no consensus state
I20260924 00:20:55.751036 10779 tablet_service.cc:1467] Processing CreateTablet for tablet 746fb7b4571f428a80495f6e9c262a7f (DEFAULT_TABLE table=default.Table_05ea79d978c74171980cf4ffdc8dd08e [id=96e09747aa014d5bb96e5d991b38c039]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:55.751111 10373 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_05ea79d978c74171980cf4ffdc8dd08e" } modify_external_catalogs: true
I20260924 00:20:55.751621 10373 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_05ea79d978c74171980cf4ffdc8dd08e" } modify_external_catalogs: true
I20260924 00:20:55.752260 10779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 746fb7b4571f428a80495f6e9c262a7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:55.753471 11051 log.cc:1196] T 84a455f7221148de87e2c3a5995afd5d P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/84a455f7221148de87e2c3a5995afd5d
W20260924 00:20:55.753564 10369 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 720e1a36bcfb4260b3689d99a7358205 has no consensus state
I20260924 00:20:55.754582 11051 ts_tablet_manager.cc:1939] T 84a455f7221148de87e2c3a5995afd5d P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:20:55.756850 11039 ts_tablet_manager.cc:1428] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.061s user 0.005s sys 0.000s
I20260924 00:20:55.759868 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 84a455f7221148de87e2c3a5995afd5d (table default.Table_bf986279c4534aa9bf4f06adb96f16ea [id=43419db539614db8802a84dff868f6b9]) successfully deleted
I20260924 00:20:55.765154 11040 tablet_bootstrap.cc:492] T 1e5beeac2a1d4cb18b5d3d2408fd574b P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:20:55.765668 11040 ts_tablet_manager.cc:1397] T 1e5beeac2a1d4cb18b5d3d2408fd574b P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.024s user 0.008s sys 0.008s
I20260924 00:20:55.766371 11039 tablet_bootstrap.cc:492] T e1864ca7a2e44611b6bb4e87f04dc61f P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:20:55.767138 11051 tablet_replica.cc:330] T 58d19cff0804461d9b4deca952b94386 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:20:55.767998 11051 raft_consensus.cc:2229] T 58d19cff0804461d9b4deca952b94386 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:55.768435 11051 raft_consensus.cc:2258] T 58d19cff0804461d9b4deca952b94386 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:55.770762 11051 ts_tablet_manager.cc:1905] T 58d19cff0804461d9b4deca952b94386 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:55.768326 11040 raft_consensus.cc:348] T 1e5beeac2a1d4cb18b5d3d2408fd574b P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:55.749658 10920 tablet_service.cc:1467] Processing CreateTablet for tablet 746fb7b4571f428a80495f6e9c262a7f (DEFAULT_TABLE table=default.Table_05ea79d978c74171980cf4ffdc8dd08e [id=96e09747aa014d5bb96e5d991b38c039]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:55.779245 11040 raft_consensus.cc:374] T 1e5beeac2a1d4cb18b5d3d2408fd574b P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:55.779590 11040 raft_consensus.cc:729] T 1e5beeac2a1d4cb18b5d3d2408fd574b P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:20:55.779989 10920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 746fb7b4571f428a80495f6e9c262a7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:55.780350 11040 consensus_queue.cc:260] T 1e5beeac2a1d4cb18b5d3d2408fd574b P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:55.783653 10369 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_2d9dbcf42731420199dabcdd3f6df616" } modify_external_catalogs: true
I20260924 00:20:55.784230 10369 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_2d9dbcf42731420199dabcdd3f6df616" } modify_external_catalogs: true
I20260924 00:20:55.785181 11040 ts_tablet_manager.cc:1428] T 1e5beeac2a1d4cb18b5d3d2408fd574b P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.019s user 0.002s sys 0.004s
I20260924 00:20:55.785710 10777 tablet_service.cc:1467] Processing CreateTablet for tablet ad256de63e0948928a51d9231152ac2d (DEFAULT_TABLE table=default.Table_0a635e42c38a4b84a1ac9d8838333c9a [id=f6322db8cfe84fcfbaa47bb7a17e492e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:55.786023 11040 tablet_bootstrap.cc:492] T 8fc17bdba09b40b7a3280c64121ff508 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:20:55.786716 10777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad256de63e0948928a51d9231152ac2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:55.789040 11041 tablet_bootstrap.cc:492] T 1e5beeac2a1d4cb18b5d3d2408fd574b P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:20:55.789865 11041 ts_tablet_manager.cc:1397] T 1e5beeac2a1d4cb18b5d3d2408fd574b P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.082s user 0.012s sys 0.012s
I20260924 00:20:55.792341 11041 raft_consensus.cc:348] T 1e5beeac2a1d4cb18b5d3d2408fd574b P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:55.793103 11041 raft_consensus.cc:374] T 1e5beeac2a1d4cb18b5d3d2408fd574b P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:55.793417 11041 raft_consensus.cc:729] T 1e5beeac2a1d4cb18b5d3d2408fd574b P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:20:55.794142 11041 consensus_queue.cc:260] T 1e5beeac2a1d4cb18b5d3d2408fd574b P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:55.799029 11040 tablet_bootstrap.cc:654] T 8fc17bdba09b40b7a3280c64121ff508 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:55.799584 11041 ts_tablet_manager.cc:1428] T 1e5beeac2a1d4cb18b5d3d2408fd574b P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.009s user 0.001s sys 0.004s
I20260924 00:20:55.806375 11051 ts_tablet_manager.cc:1918] T 58d19cff0804461d9b4deca952b94386 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:55.807456 10374 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 61c5ea5aefa3448ca62abe8bb6eec2e8
I20260924 00:20:55.808801 10374 catalog_manager.cc:6565] ReplaceTablet: tablet 61c5ea5aefa3448ca62abe8bb6eec2e8 deleted and replaced by tablet 5f84857aad964b01b94e69af13007a55
I20260924 00:20:55.808995 10780 tablet_service.cc:1514] Processing DeleteTablet for tablet 61c5ea5aefa3448ca62abe8bb6eec2e8 with delete_type TABLET_DATA_DELETED (replaced by tablet 5f84857aad964b01b94e69af13007a55) from {username='slave'} at 127.0.0.1:35996
I20260924 00:20:55.810105 11050 tablet_replica.cc:330] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:20:55.810750 11050 raft_consensus.cc:2229] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:55.811141 11050 raft_consensus.cc:2258] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:55.811443 10633 tablet_service.cc:1514] Processing DeleteTablet for tablet 61c5ea5aefa3448ca62abe8bb6eec2e8 with delete_type TABLET_DATA_DELETED (replaced by tablet 5f84857aad964b01b94e69af13007a55) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:55.812095 11049 tablet_replica.cc:330] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:20:55.812261 11041 tablet_bootstrap.cc:492] T 8fc17bdba09b40b7a3280c64121ff508 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:20:55.812722 11049 raft_consensus.cc:2229] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:55.813176 11049 raft_consensus.cc:2258] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:55.813862 11050 ts_tablet_manager.cc:1905] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:55.815152 11049 ts_tablet_manager.cc:1905] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:55.817720 11041 tablet_bootstrap.cc:654] T 8fc17bdba09b40b7a3280c64121ff508 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:55.825925 10922 tablet_service.cc:1514] Processing DeleteTablet for tablet 61c5ea5aefa3448ca62abe8bb6eec2e8 with delete_type TABLET_DATA_DELETED (replaced by tablet 5f84857aad964b01b94e69af13007a55) from {username='slave'} at 127.0.0.1:35358
I20260924 00:20:55.807539 11051 log.cc:1196] T 58d19cff0804461d9b4deca952b94386 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/58d19cff0804461d9b4deca952b94386
I20260924 00:20:55.833590 11045 raft_consensus.cc:482] T 1e5beeac2a1d4cb18b5d3d2408fd574b P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:20:55.834131 11045 raft_consensus.cc:504] T 1e5beeac2a1d4cb18b5d3d2408fd574b P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:55.836077 11051 ts_tablet_manager.cc:1939] T 58d19cff0804461d9b4deca952b94386 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:20:55.838922 11051 tablet_replica.cc:330] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:20:55.839125 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 58d19cff0804461d9b4deca952b94386 (table default.Table_d4b54eed2b8a4b3994b2635e01c5c273 [id=143547db8d2d4517877751965d3042ce]) successfully deleted
I20260924 00:20:55.839543 11051 raft_consensus.cc:2229] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus shutting down.
W20260924 00:20:55.839818 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet e1864ca7a2e44611b6bb4e87f04dc61f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e1864ca7a2e44611b6bb4e87f04dc61f already in progress: creating tablet
I20260924 00:20:55.839980 11051 raft_consensus.cc:2258] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:55.840533 10369 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 971e0913f7454a4a9ec25ac3891d2b1e
I20260924 00:20:55.841138 10373 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 746fb7b4571f428a80495f6e9c262a7f
I20260924 00:20:55.844189 10780 tablet_service.cc:1514] Processing DeleteTablet for tablet 971e0913f7454a4a9ec25ac3891d2b1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:55 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:20:55.844821 10778 tablet_service.cc:1514] Processing DeleteTablet for tablet 746fb7b4571f428a80495f6e9c262a7f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:55 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:20:55.846534 11050 ts_tablet_manager.cc:1918] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:55.846889 11050 log.cc:1196] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/61c5ea5aefa3448ca62abe8bb6eec2e8
I20260924 00:20:55.847525 11050 ts_tablet_manager.cc:1939] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:20:55.847888 10921 tablet_service.cc:1514] Processing DeleteTablet for tablet 746fb7b4571f428a80495f6e9c262a7f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:55 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:20:55.849973 10633 tablet_service.cc:1514] Processing DeleteTablet for tablet 971e0913f7454a4a9ec25ac3891d2b1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:55 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:55.850631 10636 tablet_service.cc:1514] Processing DeleteTablet for tablet 746fb7b4571f428a80495f6e9c262a7f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:55 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:55.852386 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 61c5ea5aefa3448ca62abe8bb6eec2e8 (table default.Table_d4b54eed2b8a4b3994b2635e01c5c273 [id=143547db8d2d4517877751965d3042ce]) successfully deleted
W20260924 00:20:55.853157 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 971e0913f7454a4a9ec25ac3891d2b1e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 971e0913f7454a4a9ec25ac3891d2b1e already in progress: creating tablet
W20260924 00:20:55.853821 10361 catalog_manager.cc:4808] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 746fb7b4571f428a80495f6e9c262a7f because the tablet was not found. No further retry: Not found: Tablet not found: 746fb7b4571f428a80495f6e9c262a7f
I20260924 00:20:55.862439 11045 leader_election.cc:288] T 1e5beeac2a1d4cb18b5d3d2408fd574b P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:20:55.865085 10922 tablet_service.cc:1514] Processing DeleteTablet for tablet 971e0913f7454a4a9ec25ac3891d2b1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:55 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:20:55.870474 10373 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_f03681b15e3c48d3a75aac07e44bded8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:20:55.872062 10373 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f03681b15e3c48d3a75aac07e44bded8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:55.881363 10369 master_service.cc:946] ReplaceTablet: received request to replace tablet 746fb7b4571f428a80495f6e9c262a7f from {username='slave'} at 127.0.0.1:47022
I20260924 00:20:55.857378 11039 tablet_bootstrap.cc:654] T e1864ca7a2e44611b6bb4e87f04dc61f P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:55.887041 10778 tablet_service.cc:1467] Processing CreateTablet for tablet 720e1a36bcfb4260b3689d99a7358205 (DEFAULT_TABLE table=default.Table_b1ae3ebb4b3e4c43a1a684b765cfdd38 [id=9d8bc2174e884f97b364cc1d2e5af3b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:55.888074 10778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 720e1a36bcfb4260b3689d99a7358205. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:55.857882 11051 ts_tablet_manager.cc:1905] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:55.889987 10636 tablet_service.cc:1467] Processing CreateTablet for tablet 720e1a36bcfb4260b3689d99a7358205 (DEFAULT_TABLE table=default.Table_b1ae3ebb4b3e4c43a1a684b765cfdd38 [id=9d8bc2174e884f97b364cc1d2e5af3b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:55.891012 10636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 720e1a36bcfb4260b3689d99a7358205. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:55.890885 10780 tablet_service.cc:1467] Processing CreateTablet for tablet a1a79421de4049aea258261ff9bd00b4 (DEFAULT_TABLE table=default.Table_b1726c09065e4fbaa2e6799b3ebdbc74 [id=4dbb68e0a32349ad8da2945db650584d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:55.891805 10780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1a79421de4049aea258261ff9bd00b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:55.891937 10633 tablet_service.cc:1467] Processing CreateTablet for tablet a1a79421de4049aea258261ff9bd00b4 (DEFAULT_TABLE table=default.Table_b1726c09065e4fbaa2e6799b3ebdbc74 [id=4dbb68e0a32349ad8da2945db650584d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:55.892849 10633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1a79421de4049aea258261ff9bd00b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:55.899819 10370 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_fe6a80965042459785c87b1276c5c7fe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:20:55.901372 10370 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fe6a80965042459785c87b1276c5c7fe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:55.904775 10368 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_ce92a0828938401292681552add70c45"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:20:55.908376 10776 tablet_service.cc:1514] Processing DeleteTablet for tablet 61c5ea5aefa3448ca62abe8bb6eec2e8 with delete_type TABLET_DATA_DELETED (replaced by tablet 5f84857aad964b01b94e69af13007a55) from {username='slave'} at 127.0.0.1:35996
W20260924 00:20:55.909833 10361 catalog_manager.cc:4808] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 61c5ea5aefa3448ca62abe8bb6eec2e8 because the tablet was not found. No further retry: Not found: Tablet not found: 61c5ea5aefa3448ca62abe8bb6eec2e8
W20260924 00:20:55.906306 10368 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ce92a0828938401292681552add70c45 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:55.929361 10656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e5beeac2a1d4cb18b5d3d2408fd574b" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:20:55.930213 10656 raft_consensus.cc:2454] T 1e5beeac2a1d4cb18b5d3d2408fd574b P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 0.
I20260924 00:20:55.931713 10733 leader_election.cc:302] T 1e5beeac2a1d4cb18b5d3d2408fd574b P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670; no voters:
I20260924 00:20:55.932726 11045 raft_consensus.cc:2781] T 1e5beeac2a1d4cb18b5d3d2408fd574b P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:20:55.933164 11045 raft_consensus.cc:482] T 1e5beeac2a1d4cb18b5d3d2408fd574b P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:20:55.933530 11045 raft_consensus.cc:3037] T 1e5beeac2a1d4cb18b5d3d2408fd574b P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:20:55.951853 10373 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 1e5beeac2a1d4cb18b5d3d2408fd574b (table default.Table_6f33bd5289bf46dcb45c4abf247b61c1 [id=30a3f7c8fb9c46cf8b883eedae25d938]). Expected version 1 got 0
I20260924 00:20:55.953845 11045 raft_consensus.cc:504] T 1e5beeac2a1d4cb18b5d3d2408fd574b P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:55.955957 10371 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet 1e5beeac2a1d4cb18b5d3d2408fd574b (table default.Table_6f33bd5289bf46dcb45c4abf247b61c1 [id=30a3f7c8fb9c46cf8b883eedae25d938]). Expected version 1 got 0
I20260924 00:20:55.956172 10776 tablet_service.cc:1467] Processing CreateTablet for tablet 5f84857aad964b01b94e69af13007a55 (DEFAULT_TABLE table=default.Table_d4b54eed2b8a4b3994b2635e01c5c273 [id=143547db8d2d4517877751965d3042ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:55.957340 10776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f84857aad964b01b94e69af13007a55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:55.958853 10656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e5beeac2a1d4cb18b5d3d2408fd574b" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:20:55.959390 10656 raft_consensus.cc:3037] T 1e5beeac2a1d4cb18b5d3d2408fd574b P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:20:55.962699 10775 tablet_service.cc:1467] Processing CreateTablet for tablet 9a288e9b6f804283b6d48e4d4e1dd828 (DEFAULT_TABLE table=default.Table_5f8c295237ab4e0ea184a03a9fa458a2 [id=be698d8b82734ff5940f6915ba1d4df1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:55.964202 10775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a288e9b6f804283b6d48e4d4e1dd828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:55.964242 10921 tablet_service.cc:1467] Processing CreateTablet for tablet a1a79421de4049aea258261ff9bd00b4 (DEFAULT_TABLE table=default.Table_b1726c09065e4fbaa2e6799b3ebdbc74 [id=4dbb68e0a32349ad8da2945db650584d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:55.964815 10371 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_6f33bd5289bf46dcb45c4abf247b61c1" } new_table_name: "default.Table_e4e86616e1c84fd9826d18eed1f12758" modify_external_catalogs: true
I20260924 00:20:55.965250 10921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1a79421de4049aea258261ff9bd00b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:55.965969 10656 raft_consensus.cc:2454] T 1e5beeac2a1d4cb18b5d3d2408fd574b P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 1.
I20260924 00:20:55.967160 10373 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_fe6a80965042459785c87b1276c5c7fe" } new_table_name: "default.Table_dd9f87be029b4087b002975c2067d2f3" modify_external_catalogs: true
I20260924 00:20:55.966887 10733 leader_election.cc:302] T 1e5beeac2a1d4cb18b5d3d2408fd574b P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670; no voters:
I20260924 00:20:55.972081 10922 tablet_service.cc:1467] Processing CreateTablet for tablet 720e1a36bcfb4260b3689d99a7358205 (DEFAULT_TABLE table=default.Table_b1ae3ebb4b3e4c43a1a684b765cfdd38 [id=9d8bc2174e884f97b364cc1d2e5af3b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:55.973196 10922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 720e1a36bcfb4260b3689d99a7358205. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:55.975765 10633 tablet_service.cc:1467] Processing CreateTablet for tablet 5f84857aad964b01b94e69af13007a55 (DEFAULT_TABLE table=default.Table_d4b54eed2b8a4b3994b2635e01c5c273 [id=143547db8d2d4517877751965d3042ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:55.976759 10633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f84857aad964b01b94e69af13007a55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:55.978814 10917 tablet_service.cc:1467] Processing CreateTablet for tablet 9a288e9b6f804283b6d48e4d4e1dd828 (DEFAULT_TABLE table=default.Table_5f8c295237ab4e0ea184a03a9fa458a2 [id=be698d8b82734ff5940f6915ba1d4df1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:55.979902 11049 ts_tablet_manager.cc:1918] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:55.980266 11049 log.cc:1196] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/61c5ea5aefa3448ca62abe8bb6eec2e8
I20260924 00:20:55.980623 10917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a288e9b6f804283b6d48e4d4e1dd828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:55.987957 10635 tablet_service.cc:1467] Processing CreateTablet for tablet 9a288e9b6f804283b6d48e4d4e1dd828 (DEFAULT_TABLE table=default.Table_5f8c295237ab4e0ea184a03a9fa458a2 [id=be698d8b82734ff5940f6915ba1d4df1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:55.988540 11045 leader_election.cc:288] T 1e5beeac2a1d4cb18b5d3d2408fd574b P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Requested vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:20:55.989034 10635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a288e9b6f804283b6d48e4d4e1dd828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:55.989193 11045 raft_consensus.cc:2781] T 1e5beeac2a1d4cb18b5d3d2408fd574b P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:20:55.989615 11045 raft_consensus.cc:686] T 1e5beeac2a1d4cb18b5d3d2408fd574b P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: Becoming Leader. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Running, Role: LEADER
I20260924 00:20:55.990445 11045 consensus_queue.cc:237] T 1e5beeac2a1d4cb18b5d3d2408fd574b P 48fbd603abe146c585d15196e1b5be08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:55.998054 11040 tablet_bootstrap.cc:492] T 8fc17bdba09b40b7a3280c64121ff508 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:20:55.998489 11040 ts_tablet_manager.cc:1397] T 8fc17bdba09b40b7a3280c64121ff508 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.213s user 0.013s sys 0.015s
I20260924 00:20:56.001559 11040 raft_consensus.cc:348] T 8fc17bdba09b40b7a3280c64121ff508 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:56.002298 11040 raft_consensus.cc:374] T 8fc17bdba09b40b7a3280c64121ff508 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:56.002630 11040 raft_consensus.cc:729] T 8fc17bdba09b40b7a3280c64121ff508 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:20:56.002632 10918 tablet_service.cc:1467] Processing CreateTablet for tablet 5f84857aad964b01b94e69af13007a55 (DEFAULT_TABLE table=default.Table_d4b54eed2b8a4b3994b2635e01c5c273 [id=143547db8d2d4517877751965d3042ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:56.003435 11040 consensus_queue.cc:260] T 8fc17bdba09b40b7a3280c64121ff508 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:56.007309 10257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu
/tmp/dist-test-taskko7HPF/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.4.124:35345
--webserver_interface=127.10.4.124
--webserver_port=35831
--builtin_ntp_servers=127.10.4.84:38335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.4.126:34479,127.10.4.125:34487,127.10.4.124:35345
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260924 00:20:55.980890 11049 ts_tablet_manager.cc:1939] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:20:56.016419 10368 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_3b2077a1a0a446f98a6ddb235c703345"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:20:56.017882 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 61c5ea5aefa3448ca62abe8bb6eec2e8 (table default.Table_d4b54eed2b8a4b3994b2635e01c5c273 [id=143547db8d2d4517877751965d3042ce]) successfully deleted
W20260924 00:20:56.018615 10368 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3b2077a1a0a446f98a6ddb235c703345 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260924 00:20:56.019837 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 971e0913f7454a4a9ec25ac3891d2b1e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 971e0913f7454a4a9ec25ac3891d2b1e already in progress: creating tablet
W20260924 00:20:56.021903 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 746fb7b4571f428a80495f6e9c262a7f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 746fb7b4571f428a80495f6e9c262a7f already in progress: creating tablet
I20260924 00:20:56.025354 10918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f84857aad964b01b94e69af13007a55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:56.025815 11041 tablet_bootstrap.cc:492] T 8fc17bdba09b40b7a3280c64121ff508 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:20:56.026357 11041 ts_tablet_manager.cc:1397] T 8fc17bdba09b40b7a3280c64121ff508 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.214s user 0.019s sys 0.048s
I20260924 00:20:56.029256 11041 raft_consensus.cc:348] T 8fc17bdba09b40b7a3280c64121ff508 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
W20260924 00:20:56.029781 10373 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b87bc7d0b65e4153aa649444fcc2c128 has no consensus state
I20260924 00:20:56.029953 11041 raft_consensus.cc:374] T 8fc17bdba09b40b7a3280c64121ff508 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:56.030195 11041 raft_consensus.cc:729] T 8fc17bdba09b40b7a3280c64121ff508 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:20:56.031412 11041 consensus_queue.cc:260] T 8fc17bdba09b40b7a3280c64121ff508 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
W20260924 00:20:56.031957 10371 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1e5beeac2a1d4cb18b5d3d2408fd574b consensus state has no leader
I20260924 00:20:56.033317 11041 ts_tablet_manager.cc:1428] T 8fc17bdba09b40b7a3280c64121ff508 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.007s user 0.005s sys 0.002s
I20260924 00:20:56.034098 11041 tablet_bootstrap.cc:492] T e1864ca7a2e44611b6bb4e87f04dc61f P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:20:56.039783 11041 tablet_bootstrap.cc:654] T e1864ca7a2e44611b6bb4e87f04dc61f P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:56.050597 10373 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_b1ae3ebb4b3e4c43a1a684b765cfdd38" } modify_external_catalogs: true
I20260924 00:20:56.051172 10373 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_b1ae3ebb4b3e4c43a1a684b765cfdd38" } modify_external_catalogs: true
I20260924 00:20:56.052486 10370 master_service.cc:946] ReplaceTablet: received request to replace tablet 720e1a36bcfb4260b3689d99a7358205 from {username='slave'} at 127.0.0.1:47022
I20260924 00:20:56.050685 10369 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_ce92a0828938401292681552add70c45" } modify_external_catalogs: true
I20260924 00:20:56.053241 10369 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_ce92a0828938401292681552add70c45" } modify_external_catalogs: true
I20260924 00:20:56.066871 10633 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fc17bdba09b40b7a3280c64121ff508 with delete_type TABLET_DATA_DELETED (Sending initial creation of tablet) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:56.071450 11049 tablet_replica.cc:330] T 8fc17bdba09b40b7a3280c64121ff508 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:20:56.072155 11049 raft_consensus.cc:2229] T 8fc17bdba09b40b7a3280c64121ff508 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:56.072587 11049 raft_consensus.cc:2258] T 8fc17bdba09b40b7a3280c64121ff508 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:56.074687 11049 ts_tablet_manager.cc:1905] T 8fc17bdba09b40b7a3280c64121ff508 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:56.075774 10635 tablet_service.cc:1467] Processing CreateTablet for tablet b87bc7d0b65e4153aa649444fcc2c128 (DEFAULT_TABLE table=default.Table_dd9f87be029b4087b002975c2067d2f3 [id=87d4d63333db40a389607571eb2171ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:56.077267 10635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b87bc7d0b65e4153aa649444fcc2c128. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:56.090047 10373 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 720e1a36bcfb4260b3689d99a7358205
I20260924 00:20:56.098120 10370 master_service.cc:946] ReplaceTablet: received request to replace tablet 9a288e9b6f804283b6d48e4d4e1dd828 from {username='slave'} at 127.0.0.1:47022
I20260924 00:20:56.103026 10778 tablet_service.cc:1514] Processing DeleteTablet for tablet 720e1a36bcfb4260b3689d99a7358205 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:56 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:20:56.106278 11049 ts_tablet_manager.cc:1918] T 8fc17bdba09b40b7a3280c64121ff508 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:56.106827 11049 log.cc:1196] T 8fc17bdba09b40b7a3280c64121ff508 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/8fc17bdba09b40b7a3280c64121ff508
I20260924 00:20:56.107549 11049 ts_tablet_manager.cc:1939] T 8fc17bdba09b40b7a3280c64121ff508 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:20:56.096989 10780 tablet_service.cc:1467] Processing CreateTablet for tablet b87bc7d0b65e4153aa649444fcc2c128 (DEFAULT_TABLE table=default.Table_dd9f87be029b4087b002975c2067d2f3 [id=87d4d63333db40a389607571eb2171ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:56.113207 10371 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_8a1236df8c9b4f3cb466498eae8f4be9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:20:56.114744 10371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8a1236df8c9b4f3cb466498eae8f4be9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:56.116343 11041 tablet_bootstrap.cc:492] T e1864ca7a2e44611b6bb4e87f04dc61f P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:20:56.116758 11041 ts_tablet_manager.cc:1397] T e1864ca7a2e44611b6bb4e87f04dc61f P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.083s user 0.013s sys 0.004s
I20260924 00:20:56.111521 10774 tablet_service.cc:1467] Processing CreateTablet for tablet c83942a74816465fa7f3f2c3e18c98bb (DEFAULT_TABLE table=default.Table_f03681b15e3c48d3a75aac07e44bded8 [id=7eb84ebbe346450eb2b944db685c414b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:56.120090 11041 raft_consensus.cc:348] T e1864ca7a2e44611b6bb4e87f04dc61f P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:56.120854 11041 raft_consensus.cc:374] T e1864ca7a2e44611b6bb4e87f04dc61f P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:56.121186 11041 raft_consensus.cc:729] T e1864ca7a2e44611b6bb4e87f04dc61f P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:20:56.122570 10633 tablet_service.cc:1467] Processing CreateTablet for tablet c83942a74816465fa7f3f2c3e18c98bb (DEFAULT_TABLE table=default.Table_f03681b15e3c48d3a75aac07e44bded8 [id=7eb84ebbe346450eb2b944db685c414b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:56.122397 11041 consensus_queue.cc:260] T e1864ca7a2e44611b6bb4e87f04dc61f P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:56.123603 10633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c83942a74816465fa7f3f2c3e18c98bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:56.128928 11041 ts_tablet_manager.cc:1428] T e1864ca7a2e44611b6bb4e87f04dc61f P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20260924 00:20:56.130124 10636 tablet_service.cc:1514] Processing DeleteTablet for tablet 720e1a36bcfb4260b3689d99a7358205 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:56 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:56.131876 10636 tablet_service.cc:1467] Processing CreateTablet for tablet 732a45a824274bdda8f7c060723a8cd2 (DEFAULT_TABLE table=default.Table_ce92a0828938401292681552add70c45 [id=18a2080b0c7344a28df81315ee1e5cc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:56.132844 10636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 732a45a824274bdda8f7c060723a8cd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:56.133750 11041 tablet_bootstrap.cc:492] T 971e0913f7454a4a9ec25ac3891d2b1e P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:20:56.142057 11046 raft_consensus.cc:482] T e1864ca7a2e44611b6bb4e87f04dc61f P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:20:56.142549 11046 raft_consensus.cc:504] T e1864ca7a2e44611b6bb4e87f04dc61f P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:56.145498 10374 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_d39fffcfdad94ae080bcf81a608b2424"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:20:56.145897 10372 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_2d4af2cf18544b6e94ef78ac7dd3bcce"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:20:56.147399 10372 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2d4af2cf18544b6e94ef78ac7dd3bcce in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260924 00:20:56.147629 10374 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d39fffcfdad94ae080bcf81a608b2424 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:56.149842 10920 tablet_service.cc:1467] Processing CreateTablet for tablet c83942a74816465fa7f3f2c3e18c98bb (DEFAULT_TABLE table=default.Table_f03681b15e3c48d3a75aac07e44bded8 [id=7eb84ebbe346450eb2b944db685c414b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:56.150645 10917 tablet_service.cc:1467] Processing CreateTablet for tablet b87bc7d0b65e4153aa649444fcc2c128 (DEFAULT_TABLE table=default.Table_dd9f87be029b4087b002975c2067d2f3 [id=87d4d63333db40a389607571eb2171ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:56.151546 10919 tablet_service.cc:1514] Processing DeleteTablet for tablet 720e1a36bcfb4260b3689d99a7358205 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:56 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:20:56.151675 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 8fc17bdba09b40b7a3280c64121ff508 (table default.Table_bf986279c4534aa9bf4f06adb96f16ea [id=43419db539614db8802a84dff868f6b9]) successfully deleted
W20260924 00:20:56.152320 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 720e1a36bcfb4260b3689d99a7358205 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 720e1a36bcfb4260b3689d99a7358205 already in progress: creating tablet
I20260924 00:20:56.157439 11046 leader_election.cc:288] T e1864ca7a2e44611b6bb4e87f04dc61f P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:20:56.163110 10634 tablet_service.cc:1514] Processing DeleteTablet for tablet e1864ca7a2e44611b6bb4e87f04dc61f with delete_type TABLET_DATA_DELETED (replaced by tablet 720e1a36bcfb4260b3689d99a7358205) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:56.163838 11049 tablet_replica.cc:330] T e1864ca7a2e44611b6bb4e87f04dc61f P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:20:56.164475 11049 raft_consensus.cc:2229] T e1864ca7a2e44611b6bb4e87f04dc61f P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:56.164898 11049 raft_consensus.cc:2258] T e1864ca7a2e44611b6bb4e87f04dc61f P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:56.165210 10778 tablet_service.cc:1467] Processing CreateTablet for tablet 732a45a824274bdda8f7c060723a8cd2 (DEFAULT_TABLE table=default.Table_ce92a0828938401292681552add70c45 [id=18a2080b0c7344a28df81315ee1e5cc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:56.166697 11041 tablet_bootstrap.cc:654] T 971e0913f7454a4a9ec25ac3891d2b1e P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:56.167027 11049 ts_tablet_manager.cc:1905] T e1864ca7a2e44611b6bb4e87f04dc61f P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:56.173903 10780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b87bc7d0b65e4153aa649444fcc2c128. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:56.177598 10778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 732a45a824274bdda8f7c060723a8cd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:56.185218 10369 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 732a45a824274bdda8f7c060723a8cd2
I20260924 00:20:56.185534 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c83942a74816465fa7f3f2c3e18c98bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:56.189591 10634 tablet_service.cc:1514] Processing DeleteTablet for tablet 732a45a824274bdda8f7c060723a8cd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:56 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:56.210678 10919 tablet_service.cc:1514] Processing DeleteTablet for tablet 732a45a824274bdda8f7c060723a8cd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:56 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:20:56.208624 10920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c83942a74816465fa7f3f2c3e18c98bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:56.214612 10916 tablet_service.cc:1467] Processing CreateTablet for tablet 732a45a824274bdda8f7c060723a8cd2 (DEFAULT_TABLE table=default.Table_ce92a0828938401292681552add70c45 [id=18a2080b0c7344a28df81315ee1e5cc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:56.208886 11051 ts_tablet_manager.cc:1918] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:56.221674 11051 log.cc:1196] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/61c5ea5aefa3448ca62abe8bb6eec2e8
I20260924 00:20:56.229815 11051 ts_tablet_manager.cc:1939] T 61c5ea5aefa3448ca62abe8bb6eec2e8 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:20:56.230551 11039 tablet_bootstrap.cc:492] T e1864ca7a2e44611b6bb4e87f04dc61f P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:20:56.230947 11039 ts_tablet_manager.cc:1397] T e1864ca7a2e44611b6bb4e87f04dc61f P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.465s user 0.013s sys 0.001s
I20260924 00:20:56.231827 10917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b87bc7d0b65e4153aa649444fcc2c128. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:56.233640 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 61c5ea5aefa3448ca62abe8bb6eec2e8 (table default.Table_d4b54eed2b8a4b3994b2635e01c5c273 [id=143547db8d2d4517877751965d3042ce]) successfully deleted
W20260924 00:20:56.234403 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 746fb7b4571f428a80495f6e9c262a7f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 746fb7b4571f428a80495f6e9c262a7f already in progress: creating tablet
W20260924 00:20:56.235035 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 971e0913f7454a4a9ec25ac3891d2b1e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 971e0913f7454a4a9ec25ac3891d2b1e already in progress: creating tablet
W20260924 00:20:56.236099 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 720e1a36bcfb4260b3689d99a7358205 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 720e1a36bcfb4260b3689d99a7358205 already in progress: creating tablet
I20260924 00:20:56.237259 10916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 732a45a824274bdda8f7c060723a8cd2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:20:56.241319 10358 catalog_manager.cc:4808] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 732a45a824274bdda8f7c060723a8cd2 because the tablet was not found. No further retry: Not found: Tablet not found: 732a45a824274bdda8f7c060723a8cd2
I20260924 00:20:56.241842 10367 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_f03681b15e3c48d3a75aac07e44bded8" } new_table_name: "default.Table_fe46adb213e84b79beecb3d7e78707c1" modify_external_catalogs: true
I20260924 00:20:56.242200 10369 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_e4e86616e1c84fd9826d18eed1f12758" } new_table_name: "default.Table_8d75520690044effa0a919d3715e4a2f" modify_external_catalogs: true
I20260924 00:20:56.245119 11039 raft_consensus.cc:348] T e1864ca7a2e44611b6bb4e87f04dc61f P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:56.245944 11039 raft_consensus.cc:374] T e1864ca7a2e44611b6bb4e87f04dc61f P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:56.246415 11039 raft_consensus.cc:729] T e1864ca7a2e44611b6bb4e87f04dc61f P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
W20260924 00:20:56.249244 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 720e1a36bcfb4260b3689d99a7358205 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 720e1a36bcfb4260b3689d99a7358205 already in progress: creating tablet
I20260924 00:20:56.249888 10775 tablet_service.cc:1514] Processing DeleteTablet for tablet 732a45a824274bdda8f7c060723a8cd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:56 UTC) from {username='slave'} at 127.0.0.1:35996
W20260924 00:20:56.251158 10361 catalog_manager.cc:4808] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 732a45a824274bdda8f7c060723a8cd2 because the tablet was not found. No further retry: Not found: Tablet not found: 732a45a824274bdda8f7c060723a8cd2
I20260924 00:20:56.251693 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e5beeac2a1d4cb18b5d3d2408fd574b" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:20:56.253441 10941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e5beeac2a1d4cb18b5d3d2408fd574b" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef"
I20260924 00:20:56.254369 11039 consensus_queue.cc:260] T e1864ca7a2e44611b6bb4e87f04dc61f P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
W20260924 00:20:56.254887 10732 leader_election.cc:341] T 1e5beeac2a1d4cb18b5d3d2408fd574b P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
W20260924 00:20:56.255918 10732 leader_election.cc:341] T 1e5beeac2a1d4cb18b5d3d2408fd574b P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:20:56.266637 11040 ts_tablet_manager.cc:1428] T 8fc17bdba09b40b7a3280c64121ff508 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.268s user 0.023s sys 0.002s
I20260924 00:20:56.284744 11040 tablet_bootstrap.cc:492] T e1864ca7a2e44611b6bb4e87f04dc61f P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
W20260924 00:20:56.292048 10733 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:33461, user_credentials={real_user=slave}} blocked reactor thread for 325200us
I20260924 00:20:56.296108 10370 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 9a288e9b6f804283b6d48e4d4e1dd828
I20260924 00:20:56.297591 10370 catalog_manager.cc:6565] ReplaceTablet: tablet 9a288e9b6f804283b6d48e4d4e1dd828 deleted and replaced by tablet 1877e10d1db845f2a001063fb3bffef4
I20260924 00:20:56.296911 10373 catalog_manager.cc:5495] T 1e5beeac2a1d4cb18b5d3d2408fd574b P 48fbd603abe146c585d15196e1b5be08 reported cstate change: term changed from 0 to 1, leader changed from <none> to 48fbd603abe146c585d15196e1b5be08 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "48fbd603abe146c585d15196e1b5be08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } health_report { overall_health: UNKNOWN } } }
I20260924 00:20:56.298640 10373 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet 1e5beeac2a1d4cb18b5d3d2408fd574b (table default.Table_e4e86616e1c84fd9826d18eed1f12758 [id=30a3f7c8fb9c46cf8b883eedae25d938]). Expected version 2 got 0
I20260924 00:20:56.310814 10775 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a288e9b6f804283b6d48e4d4e1dd828 with delete_type TABLET_DATA_DELETED (replaced by tablet 1877e10d1db845f2a001063fb3bffef4) from {username='slave'} at 127.0.0.1:35996
W20260924 00:20:56.313284 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 9a288e9b6f804283b6d48e4d4e1dd828 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9a288e9b6f804283b6d48e4d4e1dd828 already in progress: creating tablet
I20260924 00:20:56.321460 11040 tablet_bootstrap.cc:654] T e1864ca7a2e44611b6bb4e87f04dc61f P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:56.330807 11039 ts_tablet_manager.cc:1428] T e1864ca7a2e44611b6bb4e87f04dc61f P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.100s user 0.009s sys 0.019s
I20260924 00:20:56.331782 11039 tablet_bootstrap.cc:492] T 971e0913f7454a4a9ec25ac3891d2b1e P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:20:56.335095 10918 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a288e9b6f804283b6d48e4d4e1dd828 with delete_type TABLET_DATA_DELETED (replaced by tablet 1877e10d1db845f2a001063fb3bffef4) from {username='slave'} at 127.0.0.1:35358
W20260924 00:20:56.341527 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 9a288e9b6f804283b6d48e4d4e1dd828 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9a288e9b6f804283b6d48e4d4e1dd828 already in progress: creating tablet
W20260924 00:20:56.343199 10361 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 720e1a36bcfb4260b3689d99a7358205 (table default.Table_b1ae3ebb4b3e4c43a1a684b765cfdd38 [id=9d8bc2174e884f97b364cc1d2e5af3b4], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 720e1a36bcfb4260b3689d99a7358205 consensus state has no leader
I20260924 00:20:56.343434 11039 tablet_bootstrap.cc:654] T 971e0913f7454a4a9ec25ac3891d2b1e P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:56.360121 10368 master_service.cc:946] ReplaceTablet: received request to replace tablet 5f84857aad964b01b94e69af13007a55 from {username='slave'} at 127.0.0.1:47022
I20260924 00:20:56.362718 10800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1864ca7a2e44611b6bb4e87f04dc61f" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:20:56.363034 10635 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a288e9b6f804283b6d48e4d4e1dd828 with delete_type TABLET_DATA_DELETED (replaced by tablet 1877e10d1db845f2a001063fb3bffef4) from {username='slave'} at 127.0.0.1:46896
W20260924 00:20:56.364833 10592 leader_election.cc:341] T e1864ca7a2e44611b6bb4e87f04dc61f P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:20:56.369642 11040 tablet_bootstrap.cc:492] T e1864ca7a2e44611b6bb4e87f04dc61f P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:20:56.370373 11049 ts_tablet_manager.cc:1918] T e1864ca7a2e44611b6bb4e87f04dc61f P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:56.370785 11049 log.cc:1196] T e1864ca7a2e44611b6bb4e87f04dc61f P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/e1864ca7a2e44611b6bb4e87f04dc61f
I20260924 00:20:56.371439 11049 ts_tablet_manager.cc:1939] T e1864ca7a2e44611b6bb4e87f04dc61f P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:20:56.371920 11040 ts_tablet_manager.cc:1397] T e1864ca7a2e44611b6bb4e87f04dc61f P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.087s user 0.002s sys 0.010s
I20260924 00:20:56.377692 10370 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_d4b54eed2b8a4b3994b2635e01c5c273" } modify_external_catalogs: true
I20260924 00:20:56.378304 10370 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_d4b54eed2b8a4b3994b2635e01c5c273" } modify_external_catalogs: true
I20260924 00:20:56.383916 11041 tablet_bootstrap.cc:492] T 971e0913f7454a4a9ec25ac3891d2b1e P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:20:56.384217 11041 ts_tablet_manager.cc:1397] T 971e0913f7454a4a9ec25ac3891d2b1e P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.251s user 0.026s sys 0.023s
I20260924 00:20:56.373143 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet e1864ca7a2e44611b6bb4e87f04dc61f (table default.Table_b1ae3ebb4b3e4c43a1a684b765cfdd38 [id=9d8bc2174e884f97b364cc1d2e5af3b4]) successfully deleted
I20260924 00:20:56.387326 11041 raft_consensus.cc:348] T 971e0913f7454a4a9ec25ac3891d2b1e P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:56.387941 11041 raft_consensus.cc:374] T 971e0913f7454a4a9ec25ac3891d2b1e P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:56.388166 11041 raft_consensus.cc:729] T 971e0913f7454a4a9ec25ac3891d2b1e P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:20:56.394158 10916 tablet_service.cc:1514] Processing DeleteTablet for tablet e1864ca7a2e44611b6bb4e87f04dc61f with delete_type TABLET_DATA_DELETED (replaced by tablet 720e1a36bcfb4260b3689d99a7358205) from {username='slave'} at 127.0.0.1:35358
W20260924 00:20:56.394594 10367 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c83942a74816465fa7f3f2c3e18c98bb consensus state has no leader
I20260924 00:20:56.395056 11051 tablet_replica.cc:330] T e1864ca7a2e44611b6bb4e87f04dc61f P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
W20260924 00:20:56.395735 10369 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1e5beeac2a1d4cb18b5d3d2408fd574b consensus state has no leader
I20260924 00:20:56.395699 11051 raft_consensus.cc:2229] T e1864ca7a2e44611b6bb4e87f04dc61f P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:56.395300 11041 consensus_queue.cc:260] T 971e0913f7454a4a9ec25ac3891d2b1e P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:56.396121 11051 raft_consensus.cc:2258] T e1864ca7a2e44611b6bb4e87f04dc61f P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:56.403000 11041 ts_tablet_manager.cc:1428] T 971e0913f7454a4a9ec25ac3891d2b1e P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.019s user 0.002s sys 0.005s
I20260924 00:20:56.402963 11051 ts_tablet_manager.cc:1905] T e1864ca7a2e44611b6bb4e87f04dc61f P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:56.406903 11041 tablet_bootstrap.cc:492] T 746fb7b4571f428a80495f6e9c262a7f P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:20:56.416982 11041 tablet_bootstrap.cc:654] T 746fb7b4571f428a80495f6e9c262a7f P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
W20260924 00:20:56.425602 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 732a45a824274bdda8f7c060723a8cd2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 732a45a824274bdda8f7c060723a8cd2 already in progress: creating tablet
W20260924 00:20:56.427022 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 9a288e9b6f804283b6d48e4d4e1dd828 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9a288e9b6f804283b6d48e4d4e1dd828 already in progress: creating tablet
I20260924 00:20:56.429368 10941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1864ca7a2e44611b6bb4e87f04dc61f" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:20:56.430899 11039 tablet_bootstrap.cc:492] T 971e0913f7454a4a9ec25ac3891d2b1e P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:20:56.431375 11039 ts_tablet_manager.cc:1397] T 971e0913f7454a4a9ec25ac3891d2b1e P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.100s user 0.009s sys 0.035s
I20260924 00:20:56.434825 10372 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_dd9f87be029b4087b002975c2067d2f3" } modify_external_catalogs: true
I20260924 00:20:56.435400 10372 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_dd9f87be029b4087b002975c2067d2f3" } modify_external_catalogs: true
W20260924 00:20:56.437757 10589 leader_election.cc:341] T e1864ca7a2e44611b6bb4e87f04dc61f P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: Tablet not RUNNING: STOPPED
I20260924 00:20:56.438191 10589 leader_election.cc:302] T e1864ca7a2e44611b6bb4e87f04dc61f P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:20:56.434139 11039 raft_consensus.cc:348] T 971e0913f7454a4a9ec25ac3891d2b1e P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
W20260924 00:20:56.438861 10589 raft_consensus.cc:2670] T e1864ca7a2e44611b6bb4e87f04dc61f P b10a9082a37b46f09d581944914c1670: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260924 00:20:56.438891 11039 raft_consensus.cc:374] T 971e0913f7454a4a9ec25ac3891d2b1e P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:56.439260 11039 raft_consensus.cc:729] T 971e0913f7454a4a9ec25ac3891d2b1e P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:20:56.440495 11039 consensus_queue.cc:260] T 971e0913f7454a4a9ec25ac3891d2b1e P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:56.452303 11039 ts_tablet_manager.cc:1428] T 971e0913f7454a4a9ec25ac3891d2b1e P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.021s user 0.005s sys 0.001s
I20260924 00:20:56.455245 11041 tablet_bootstrap.cc:492] T 746fb7b4571f428a80495f6e9c262a7f P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:20:56.455729 11041 ts_tablet_manager.cc:1397] T 746fb7b4571f428a80495f6e9c262a7f P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.050s user 0.009s sys 0.001s
I20260924 00:20:56.456259 11039 tablet_bootstrap.cc:492] T 1e5beeac2a1d4cb18b5d3d2408fd574b P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:20:56.457787 10916 tablet_service.cc:1514] Processing DeleteTablet for tablet 971e0913f7454a4a9ec25ac3891d2b1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:55 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:20:56.458549 11041 raft_consensus.cc:348] T 746fb7b4571f428a80495f6e9c262a7f P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:56.453481 11040 raft_consensus.cc:348] T e1864ca7a2e44611b6bb4e87f04dc61f P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:56.462335 11040 raft_consensus.cc:374] T e1864ca7a2e44611b6bb4e87f04dc61f P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:56.462997 11040 raft_consensus.cc:729] T e1864ca7a2e44611b6bb4e87f04dc61f P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:20:56.464215 11041 raft_consensus.cc:374] T 746fb7b4571f428a80495f6e9c262a7f P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:56.464474 11041 raft_consensus.cc:729] T 746fb7b4571f428a80495f6e9c262a7f P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:20:56.465080 11041 consensus_queue.cc:260] T 746fb7b4571f428a80495f6e9c262a7f P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:56.466816 11041 ts_tablet_manager.cc:1428] T 746fb7b4571f428a80495f6e9c262a7f P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.011s user 0.004s sys 0.000s
I20260924 00:20:56.469144 11041 tablet_bootstrap.cc:492] T ad256de63e0948928a51d9231152ac2d P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:20:56.472743 10636 tablet_service.cc:1514] Processing DeleteTablet for tablet 971e0913f7454a4a9ec25ac3891d2b1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:55 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:56.473486 11049 tablet_replica.cc:330] T 971e0913f7454a4a9ec25ac3891d2b1e P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:20:56.474130 11049 raft_consensus.cc:2229] T 971e0913f7454a4a9ec25ac3891d2b1e P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:56.474534 11049 raft_consensus.cc:2258] T 971e0913f7454a4a9ec25ac3891d2b1e P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:56.476079 10635 tablet_service.cc:1514] Processing DeleteTablet for tablet 746fb7b4571f428a80495f6e9c262a7f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:55 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:56.478404 11049 ts_tablet_manager.cc:1905] T 971e0913f7454a4a9ec25ac3891d2b1e P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:56.479738 11041 tablet_bootstrap.cc:654] T ad256de63e0948928a51d9231152ac2d P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:56.494287 10369 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_ef16ff2bfc9b42219f0b6d5f957312f6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:20:56.496541 10369 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ef16ff2bfc9b42219f0b6d5f957312f6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:56.496790 10374 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_f7a728195d6d4f3785b7de188b9d36cd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:20:56.499122 10374 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f7a728195d6d4f3785b7de188b9d36cd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:56.512197 11051 ts_tablet_manager.cc:1918] T e1864ca7a2e44611b6bb4e87f04dc61f P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:56.464278 11040 consensus_queue.cc:260] T e1864ca7a2e44611b6bb4e87f04dc61f P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:56.514055 11040 ts_tablet_manager.cc:1428] T e1864ca7a2e44611b6bb4e87f04dc61f P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.063s user 0.004s sys 0.002s
I20260924 00:20:56.517467 10373 master_service.cc:946] ReplaceTablet: received request to replace tablet 746fb7b4571f428a80495f6e9c262a7f from {username='slave'} at 127.0.0.1:47022
I20260924 00:20:56.515893 10774 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fc17bdba09b40b7a3280c64121ff508 with delete_type TABLET_DATA_DELETED (Sending initial creation of tablet) from {username='slave'} at 127.0.0.1:35996
I20260924 00:20:56.525588 11050 tablet_replica.cc:330] T 8fc17bdba09b40b7a3280c64121ff508 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:20:56.526223 11050 raft_consensus.cc:2229] T 8fc17bdba09b40b7a3280c64121ff508 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:56.526650 11050 raft_consensus.cc:2258] T 8fc17bdba09b40b7a3280c64121ff508 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:56.528337 11050 ts_tablet_manager.cc:1905] T 8fc17bdba09b40b7a3280c64121ff508 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:56.536090 10780 tablet_service.cc:1514] Processing DeleteTablet for tablet e1864ca7a2e44611b6bb4e87f04dc61f with delete_type TABLET_DATA_DELETED (replaced by tablet 720e1a36bcfb4260b3689d99a7358205) from {username='slave'} at 127.0.0.1:35996
I20260924 00:20:56.541347 10370 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 5f84857aad964b01b94e69af13007a55
I20260924 00:20:56.542210 10372 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet b87bc7d0b65e4153aa649444fcc2c128
I20260924 00:20:56.543484 10780 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f84857aad964b01b94e69af13007a55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:56 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:20:56.545523 10780 tablet_service.cc:1514] Processing DeleteTablet for tablet b87bc7d0b65e4153aa649444fcc2c128 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:56 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:20:56.548434 10635 tablet_service.cc:1514] Processing DeleteTablet for tablet b87bc7d0b65e4153aa649444fcc2c128 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:56 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:56.548435 10636 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f84857aad964b01b94e69af13007a55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:56 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:56.561776 11040 tablet_bootstrap.cc:492] T 971e0913f7454a4a9ec25ac3891d2b1e P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:20:56.517828 11039 tablet_bootstrap.cc:654] T 1e5beeac2a1d4cb18b5d3d2408fd574b P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:56.565140 11051 log.cc:1196] T e1864ca7a2e44611b6bb4e87f04dc61f P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/e1864ca7a2e44611b6bb4e87f04dc61f
I20260924 00:20:56.561195 11041 tablet_bootstrap.cc:492] T ad256de63e0948928a51d9231152ac2d P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:20:56.565819 11051 ts_tablet_manager.cc:1939] T e1864ca7a2e44611b6bb4e87f04dc61f P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:20:56.566154 11041 ts_tablet_manager.cc:1397] T ad256de63e0948928a51d9231152ac2d P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.097s user 0.012s sys 0.001s
I20260924 00:20:56.547009 10920 tablet_service.cc:1514] Processing DeleteTablet for tablet b87bc7d0b65e4153aa649444fcc2c128 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:56 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:20:56.547178 10916 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f84857aad964b01b94e69af13007a55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:56 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:20:56.568115 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet e1864ca7a2e44611b6bb4e87f04dc61f (table default.Table_b1ae3ebb4b3e4c43a1a684b765cfdd38 [id=9d8bc2174e884f97b364cc1d2e5af3b4]) successfully deleted
I20260924 00:20:56.574714 11041 raft_consensus.cc:348] T ad256de63e0948928a51d9231152ac2d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:56.575112 11039 tablet_bootstrap.cc:492] T 1e5beeac2a1d4cb18b5d3d2408fd574b P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:20:56.575495 11039 ts_tablet_manager.cc:1397] T 1e5beeac2a1d4cb18b5d3d2408fd574b P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.119s user 0.011s sys 0.001s
I20260924 00:20:56.578295 11039 raft_consensus.cc:348] T 1e5beeac2a1d4cb18b5d3d2408fd574b P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:56.579079 11039 raft_consensus.cc:374] T 1e5beeac2a1d4cb18b5d3d2408fd574b P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260924 00:20:56.583616 10732 consensus_peers.cc:487] T 1e5beeac2a1d4cb18b5d3d2408fd574b P 48fbd603abe146c585d15196e1b5be08 -> Peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Couldn't send request to peer c8626e300d0644b0908797ad61bd7cef. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260924 00:20:56.583997 10916 tablet_service.cc:1467] Processing CreateTablet for tablet 6ad838c884fc4b1d8f98781e7190e3df (DEFAULT_TABLE table=default.Table_2d4af2cf18544b6e94ef78ac7dd3bcce [id=218c77bc322a488593d12af9a27d20a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:56.585029 10916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ad838c884fc4b1d8f98781e7190e3df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:56.585367 11051 tablet_replica.cc:330] T 971e0913f7454a4a9ec25ac3891d2b1e P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:20:56.586009 11051 raft_consensus.cc:2229] T 971e0913f7454a4a9ec25ac3891d2b1e P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:56.586409 11051 raft_consensus.cc:2258] T 971e0913f7454a4a9ec25ac3891d2b1e P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:56.583014 10636 tablet_service.cc:1467] Processing CreateTablet for tablet 6ad838c884fc4b1d8f98781e7190e3df (DEFAULT_TABLE table=default.Table_2d4af2cf18544b6e94ef78ac7dd3bcce [id=218c77bc322a488593d12af9a27d20a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:56.587756 10633 tablet_service.cc:1467] Processing CreateTablet for tablet 1877e10d1db845f2a001063fb3bffef4 (DEFAULT_TABLE table=default.Table_5f8c295237ab4e0ea184a03a9fa458a2 [id=be698d8b82734ff5940f6915ba1d4df1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:56.584419 10635 tablet_service.cc:1467] Processing CreateTablet for tablet ac559183fbef4e21b190a9a61a50e1e1 (DEFAULT_TABLE table=default.Table_3b2077a1a0a446f98a6ddb235c703345 [id=8063a759be2141ae965703f9196a18c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:56.588397 10920 tablet_service.cc:1467] Processing CreateTablet for tablet ac559183fbef4e21b190a9a61a50e1e1 (DEFAULT_TABLE table=default.Table_3b2077a1a0a446f98a6ddb235c703345 [id=8063a759be2141ae965703f9196a18c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:56.588665 10636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ad838c884fc4b1d8f98781e7190e3df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:56.589361 10920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac559183fbef4e21b190a9a61a50e1e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:56.579419 11039 raft_consensus.cc:729] T 1e5beeac2a1d4cb18b5d3d2408fd574b P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:20:56.587257 11049 ts_tablet_manager.cc:1918] T 971e0913f7454a4a9ec25ac3891d2b1e P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:56.590413 11040 tablet_bootstrap.cc:654] T 971e0913f7454a4a9ec25ac3891d2b1e P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:56.590601 11049 log.cc:1196] T 971e0913f7454a4a9ec25ac3891d2b1e P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/971e0913f7454a4a9ec25ac3891d2b1e
I20260924 00:20:56.591214 11049 ts_tablet_manager.cc:1939] T 971e0913f7454a4a9ec25ac3891d2b1e P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:20:56.593529 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 971e0913f7454a4a9ec25ac3891d2b1e (table default.Table_2d9dbcf42731420199dabcdd3f6df616 [id=e6c357c8899f430b95c9c55084999d6f]) successfully deleted
I20260924 00:20:56.593636 10656 raft_consensus.cc:1261] T 1e5beeac2a1d4cb18b5d3d2408fd574b P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Refusing update from remote peer 48fbd603abe146c585d15196e1b5be08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:20:56.595058 10635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac559183fbef4e21b190a9a61a50e1e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:56.595814 10917 tablet_service.cc:1467] Processing CreateTablet for tablet 1877e10d1db845f2a001063fb3bffef4 (DEFAULT_TABLE table=default.Table_5f8c295237ab4e0ea184a03a9fa458a2 [id=be698d8b82734ff5940f6915ba1d4df1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:56.596856 10917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1877e10d1db845f2a001063fb3bffef4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:56.597275 10633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1877e10d1db845f2a001063fb3bffef4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:56.605867 10780 tablet_service.cc:1467] Processing CreateTablet for tablet ac559183fbef4e21b190a9a61a50e1e1 (DEFAULT_TABLE table=default.Table_3b2077a1a0a446f98a6ddb235c703345 [id=8063a759be2141ae965703f9196a18c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:56.606123 10775 tablet_service.cc:1467] Processing CreateTablet for tablet 1877e10d1db845f2a001063fb3bffef4 (DEFAULT_TABLE table=default.Table_5f8c295237ab4e0ea184a03a9fa458a2 [id=be698d8b82734ff5940f6915ba1d4df1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:56.606951 11068 consensus_queue.cc:1035] T 1e5beeac2a1d4cb18b5d3d2408fd574b P 48fbd603abe146c585d15196e1b5be08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 00:20:56.609480 11039 consensus_queue.cc:260] T 1e5beeac2a1d4cb18b5d3d2408fd574b P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:56.606982 10780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac559183fbef4e21b190a9a61a50e1e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:56.611398 11039 ts_tablet_manager.cc:1428] T 1e5beeac2a1d4cb18b5d3d2408fd574b P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.036s user 0.000s sys 0.004s
I20260924 00:20:56.612244 11039 tablet_bootstrap.cc:492] T 8fc17bdba09b40b7a3280c64121ff508 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:20:56.613963 10369 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_f7a728195d6d4f3785b7de188b9d36cd" } new_table_name: "default.Table_a54d4fd8353e42a4a00cb4e5c689da7b" modify_external_catalogs: true
I20260924 00:20:56.617044 10374 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_8a1236df8c9b4f3cb466498eae8f4be9" } new_table_name: "default.Table_ca083375f0df4255b8654ed88040dfcd" modify_external_catalogs: true
I20260924 00:20:56.617293 10774 tablet_service.cc:1467] Processing CreateTablet for tablet 6ad838c884fc4b1d8f98781e7190e3df (DEFAULT_TABLE table=default.Table_2d4af2cf18544b6e94ef78ac7dd3bcce [id=218c77bc322a488593d12af9a27d20a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:56.618006 11041 raft_consensus.cc:374] T ad256de63e0948928a51d9231152ac2d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:56.618270 11041 raft_consensus.cc:729] T ad256de63e0948928a51d9231152ac2d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:20:56.618461 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ad838c884fc4b1d8f98781e7190e3df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:56.619257 11041 consensus_queue.cc:260] T ad256de63e0948928a51d9231152ac2d P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:56.620900 10368 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_3fb6b1e6699146b0aed9c1e5fa8b16f0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:20:56.621742 11050 ts_tablet_manager.cc:1918] T 8fc17bdba09b40b7a3280c64121ff508 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:56.622077 11050 log.cc:1196] T 8fc17bdba09b40b7a3280c64121ff508 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/8fc17bdba09b40b7a3280c64121ff508
I20260924 00:20:56.622763 11050 ts_tablet_manager.cc:1939] T 8fc17bdba09b40b7a3280c64121ff508 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
W20260924 00:20:56.622983 10368 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3fb6b1e6699146b0aed9c1e5fa8b16f0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:56.623596 10372 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 1e5beeac2a1d4cb18b5d3d2408fd574b (table default.Table_8d75520690044effa0a919d3715e4a2f [id=30a3f7c8fb9c46cf8b883eedae25d938]). Expected version 3 got 0
I20260924 00:20:56.625182 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 8fc17bdba09b40b7a3280c64121ff508 (table default.Table_bf986279c4534aa9bf4f06adb96f16ea [id=43419db539614db8802a84dff868f6b9]) successfully deleted
I20260924 00:20:56.630398 10775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1877e10d1db845f2a001063fb3bffef4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:56.635015 11050 tablet_replica.cc:330] T e1864ca7a2e44611b6bb4e87f04dc61f P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:20:56.635703 11050 raft_consensus.cc:2229] T e1864ca7a2e44611b6bb4e87f04dc61f P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:56.636113 11050 raft_consensus.cc:2258] T e1864ca7a2e44611b6bb4e87f04dc61f P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:56.637252 11049 tablet_replica.cc:330] T 746fb7b4571f428a80495f6e9c262a7f P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:20:56.637892 11049 raft_consensus.cc:2229] T 746fb7b4571f428a80495f6e9c262a7f P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:56.638310 11049 raft_consensus.cc:2258] T 746fb7b4571f428a80495f6e9c262a7f P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:56.642539 10373 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_391ed8efc7a54515837c1b79be85d64c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:20:56.644098 10373 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_391ed8efc7a54515837c1b79be85d64c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:56.655675 11049 ts_tablet_manager.cc:1905] T 746fb7b4571f428a80495f6e9c262a7f P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260924 00:20:56.658034 10374 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 453ce17089ce44ba8da67a0893301c3c has no consensus state
I20260924 00:20:56.662114 11050 ts_tablet_manager.cc:1905] T e1864ca7a2e44611b6bb4e87f04dc61f P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:56.667521 10374 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 1e5beeac2a1d4cb18b5d3d2408fd574b (table default.Table_8d75520690044effa0a919d3715e4a2f [id=30a3f7c8fb9c46cf8b883eedae25d938]). Expected version 3 got 0
W20260924 00:20:56.669927 10369 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e0074b1184ec4883ab1016330d98c968 has no consensus state
I20260924 00:20:56.673101 11051 ts_tablet_manager.cc:1905] T 971e0913f7454a4a9ec25ac3891d2b1e P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:56.648846 11039 tablet_bootstrap.cc:654] T 8fc17bdba09b40b7a3280c64121ff508 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:56.693684 10368 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 1e5beeac2a1d4cb18b5d3d2408fd574b (table default.Table_8d75520690044effa0a919d3715e4a2f [id=30a3f7c8fb9c46cf8b883eedae25d938]). Expected version 3 got 0
I20260924 00:20:56.695703 11039 tablet_bootstrap.cc:492] T 8fc17bdba09b40b7a3280c64121ff508 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:20:56.696102 11039 ts_tablet_manager.cc:1397] T 8fc17bdba09b40b7a3280c64121ff508 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.084s user 0.008s sys 0.001s
I20260924 00:20:56.698294 11039 raft_consensus.cc:348] T 8fc17bdba09b40b7a3280c64121ff508 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:56.698704 11041 ts_tablet_manager.cc:1428] T ad256de63e0948928a51d9231152ac2d P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.132s user 0.004s sys 0.001s
I20260924 00:20:56.698915 11039 raft_consensus.cc:374] T 8fc17bdba09b40b7a3280c64121ff508 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:56.699184 11039 raft_consensus.cc:729] T 8fc17bdba09b40b7a3280c64121ff508 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:20:56.699934 11041 tablet_bootstrap.cc:492] T a1a79421de4049aea258261ff9bd00b4 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:20:56.699790 11039 consensus_queue.cc:260] T 8fc17bdba09b40b7a3280c64121ff508 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:56.709976 11039 ts_tablet_manager.cc:1428] T 8fc17bdba09b40b7a3280c64121ff508 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.014s user 0.005s sys 0.001s
I20260924 00:20:56.712004 11049 ts_tablet_manager.cc:1918] T 746fb7b4571f428a80495f6e9c262a7f P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:56.712404 11049 log.cc:1196] T 746fb7b4571f428a80495f6e9c262a7f P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/746fb7b4571f428a80495f6e9c262a7f
I20260924 00:20:56.714924 10917 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fc17bdba09b40b7a3280c64121ff508 with delete_type TABLET_DATA_DELETED (Sending initial creation of tablet) from {username='slave'} at 127.0.0.1:35358
I20260924 00:20:56.720106 11041 tablet_bootstrap.cc:654] T a1a79421de4049aea258261ff9bd00b4 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:56.723831 11049 ts_tablet_manager.cc:1939] T 746fb7b4571f428a80495f6e9c262a7f P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:20:56.725098 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 746fb7b4571f428a80495f6e9c262a7f (table default.Table_05ea79d978c74171980cf4ffdc8dd08e [id=96e09747aa014d5bb96e5d991b38c039]) successfully deleted
W20260924 00:20:56.728389 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet b87bc7d0b65e4153aa649444fcc2c128 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b87bc7d0b65e4153aa649444fcc2c128 already in progress: creating tablet
W20260924 00:20:56.730504 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 5f84857aad964b01b94e69af13007a55 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5f84857aad964b01b94e69af13007a55 already in progress: creating tablet
I20260924 00:20:56.731163 10368 master_service.cc:946] ReplaceTablet: received request to replace tablet a1a79421de4049aea258261ff9bd00b4 from {username='slave'} at 127.0.0.1:47022
I20260924 00:20:56.734324 11041 tablet_bootstrap.cc:492] T a1a79421de4049aea258261ff9bd00b4 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:20:56.734719 11041 ts_tablet_manager.cc:1397] T a1a79421de4049aea258261ff9bd00b4 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.035s user 0.006s sys 0.005s
I20260924 00:20:56.736840 11041 raft_consensus.cc:348] T a1a79421de4049aea258261ff9bd00b4 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:56.737632 11041 raft_consensus.cc:374] T a1a79421de4049aea258261ff9bd00b4 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:56.737975 11041 raft_consensus.cc:729] T a1a79421de4049aea258261ff9bd00b4 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:20:56.739046 11041 consensus_queue.cc:260] T a1a79421de4049aea258261ff9bd00b4 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:56.740015 11039 tablet_bootstrap.cc:492] T ad256de63e0948928a51d9231152ac2d P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:20:56.747963 11041 ts_tablet_manager.cc:1428] T a1a79421de4049aea258261ff9bd00b4 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.013s user 0.005s sys 0.000s
I20260924 00:20:56.748489 10369 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_3b2077a1a0a446f98a6ddb235c703345" } modify_external_catalogs: true
I20260924 00:20:56.748847 11041 tablet_bootstrap.cc:492] T 720e1a36bcfb4260b3689d99a7358205 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:20:56.748761 10374 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_5f8c295237ab4e0ea184a03a9fa458a2" } modify_external_catalogs: true
I20260924 00:20:56.749152 10369 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_3b2077a1a0a446f98a6ddb235c703345" } modify_external_catalogs: true
I20260924 00:20:56.749346 10374 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_5f8c295237ab4e0ea184a03a9fa458a2" } modify_external_catalogs: true
I20260924 00:20:56.753168 11092 mvcc.cc:205] Tried to move back new op lower bound from 7332697114944626688 to 7332697112560857088. Current Snapshot: MvccSnapshot[applied={T|T < 7332697114944626688}]
I20260924 00:20:56.754321 11041 tablet_bootstrap.cc:654] T 720e1a36bcfb4260b3689d99a7358205 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:56.756776 10372 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_6ec498f485f04dd8a9110da631ea2824"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:20:56.758522 10372 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6ec498f485f04dd8a9110da631ea2824 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:56.769474 10370 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_99da18207fc542fbb68972724c4d410e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:20:56.757757 11098 tablet.cc:1722] T 1e5beeac2a1d4cb18b5d3d2408fd574b P 48fbd603abe146c585d15196e1b5be08: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
W20260924 00:20:56.771081 10370 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_99da18207fc542fbb68972724c4d410e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:56.775763 11039 tablet_bootstrap.cc:654] T ad256de63e0948928a51d9231152ac2d P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:56.775870 11040 tablet_bootstrap.cc:492] T 971e0913f7454a4a9ec25ac3891d2b1e P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:20:56.776291 11040 ts_tablet_manager.cc:1397] T 971e0913f7454a4a9ec25ac3891d2b1e P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.215s user 0.000s sys 0.015s
I20260924 00:20:56.778978 11097 tablet.cc:1722] T 1e5beeac2a1d4cb18b5d3d2408fd574b P b10a9082a37b46f09d581944914c1670: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260924 00:20:56.789196 10775 tablet_service.cc:1467] Processing CreateTablet for tablet ccb26e0fbc1948bc9a00e9e26c4d1139 (DEFAULT_TABLE table=default.Table_d39fffcfdad94ae080bcf81a608b2424 [id=7e9e663acedf4a1fae2f04cde97b7907]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:56.787434 10780 tablet_service.cc:1467] Processing CreateTablet for tablet c212525586544d8f9d92bc4b8bd8acaa (DEFAULT_TABLE table=default.Table_ef16ff2bfc9b42219f0b6d5f957312f6 [id=fbdec8eccace4eb3864a6cdd95795ce3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:56.790061 11040 raft_consensus.cc:348] T 971e0913f7454a4a9ec25ac3891d2b1e P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:56.790689 11040 raft_consensus.cc:374] T 971e0913f7454a4a9ec25ac3891d2b1e P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:56.790578 10917 tablet_service.cc:1467] Processing CreateTablet for tablet e0074b1184ec4883ab1016330d98c968 (DEFAULT_TABLE table=default.Table_a54d4fd8353e42a4a00cb4e5c689da7b [id=485c42c942b74abbb32bea36d25bc184]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:56.790940 11040 raft_consensus.cc:729] T 971e0913f7454a4a9ec25ac3891d2b1e P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:20:56.790804 10920 tablet_service.cc:1467] Processing CreateTablet for tablet ccb26e0fbc1948bc9a00e9e26c4d1139 (DEFAULT_TABLE table=default.Table_d39fffcfdad94ae080bcf81a608b2424 [id=7e9e663acedf4a1fae2f04cde97b7907]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:56.791603 10917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0074b1184ec4883ab1016330d98c968. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:56.792589 10775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccb26e0fbc1948bc9a00e9e26c4d1139. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:56.801412 10780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c212525586544d8f9d92bc4b8bd8acaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:56.800503 10918 tablet_service.cc:1467] Processing CreateTablet for tablet 453ce17089ce44ba8da67a0893301c3c (DEFAULT_TABLE table=default.Table_ca083375f0df4255b8654ed88040dfcd [id=f79ca8d11aa04d5f8f72d58b798ed4ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:56.805536 10371 master_service.cc:946] ReplaceTablet: received request to replace tablet 5f84857aad964b01b94e69af13007a55 from {username='slave'} at 127.0.0.1:47022
I20260924 00:20:56.807324 10778 tablet_service.cc:1467] Processing CreateTablet for tablet 453ce17089ce44ba8da67a0893301c3c (DEFAULT_TABLE table=default.Table_ca083375f0df4255b8654ed88040dfcd [id=f79ca8d11aa04d5f8f72d58b798ed4ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:56.808351 10778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 453ce17089ce44ba8da67a0893301c3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:56.809809 11046 raft_consensus.cc:482] T a1a79421de4049aea258261ff9bd00b4 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:20:56.810213 11046 raft_consensus.cc:504] T a1a79421de4049aea258261ff9bd00b4 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:56.812008 11046 leader_election.cc:288] T a1a79421de4049aea258261ff9bd00b4 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:20:56.815649 10368 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet a1a79421de4049aea258261ff9bd00b4
I20260924 00:20:56.815524 10941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1a79421de4049aea258261ff9bd00b4" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
W20260924 00:20:56.816645 10589 leader_election.cc:341] T a1a79421de4049aea258261ff9bd00b4 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:20:56.817006 10368 catalog_manager.cc:6565] ReplaceTablet: tablet a1a79421de4049aea258261ff9bd00b4 deleted and replaced by tablet 1ec36d59b5114feea144bcb72794136c
I20260924 00:20:56.816811 10800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1a79421de4049aea258261ff9bd00b4" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:20:56.818928 10776 tablet_service.cc:1514] Processing DeleteTablet for tablet a1a79421de4049aea258261ff9bd00b4 with delete_type TABLET_DATA_DELETED (replaced by tablet 1ec36d59b5114feea144bcb72794136c) from {username='slave'} at 127.0.0.1:35996
W20260924 00:20:56.819607 10592 leader_election.cc:341] T a1a79421de4049aea258261ff9bd00b4 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:20:56.819864 10634 tablet_service.cc:1467] Processing CreateTablet for tablet 453ce17089ce44ba8da67a0893301c3c (DEFAULT_TABLE table=default.Table_ca083375f0df4255b8654ed88040dfcd [id=f79ca8d11aa04d5f8f72d58b798ed4ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:56.819964 10592 leader_election.cc:302] T a1a79421de4049aea258261ff9bd00b4 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:20:56.821177 10921 tablet_service.cc:1514] Processing DeleteTablet for tablet a1a79421de4049aea258261ff9bd00b4 with delete_type TABLET_DATA_DELETED (replaced by tablet 1ec36d59b5114feea144bcb72794136c) from {username='slave'} at 127.0.0.1:35358
I20260924 00:20:56.821482 10374 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 1877e10d1db845f2a001063fb3bffef4
I20260924 00:20:56.824499 10776 tablet_service.cc:1514] Processing DeleteTablet for tablet 1877e10d1db845f2a001063fb3bffef4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:56 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:20:56.825392 10920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccb26e0fbc1948bc9a00e9e26c4d1139. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:56.825477 10367 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet 1e5beeac2a1d4cb18b5d3d2408fd574b (table default.Table_8d75520690044effa0a919d3715e4a2f [id=30a3f7c8fb9c46cf8b883eedae25d938]). Expected version 3 got 0
I20260924 00:20:56.826874 10918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 453ce17089ce44ba8da67a0893301c3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:56.829052 10921 tablet_service.cc:1514] Processing DeleteTablet for tablet 1877e10d1db845f2a001063fb3bffef4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:56 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:20:56.814635 11051 ts_tablet_manager.cc:1918] T 971e0913f7454a4a9ec25ac3891d2b1e P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:56.833201 11051 log.cc:1196] T 971e0913f7454a4a9ec25ac3891d2b1e P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/971e0913f7454a4a9ec25ac3891d2b1e
I20260924 00:20:56.833741 10774 tablet_service.cc:1467] Processing CreateTablet for tablet e0074b1184ec4883ab1016330d98c968 (DEFAULT_TABLE table=default.Table_a54d4fd8353e42a4a00cb4e5c689da7b [id=485c42c942b74abbb32bea36d25bc184]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:56.820874 10634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 453ce17089ce44ba8da67a0893301c3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:56.834395 11040 consensus_queue.cc:260] T 971e0913f7454a4a9ec25ac3891d2b1e P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:56.826591 10631 tablet_service.cc:1514] Processing DeleteTablet for tablet 1877e10d1db845f2a001063fb3bffef4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:56 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:56.835950 11040 ts_tablet_manager.cc:1428] T 971e0913f7454a4a9ec25ac3891d2b1e P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.059s user 0.003s sys 0.003s
I20260924 00:20:56.836663 11040 tablet_bootstrap.cc:492] T 720e1a36bcfb4260b3689d99a7358205 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:20:56.833828 11051 ts_tablet_manager.cc:1939] T 971e0913f7454a4a9ec25ac3891d2b1e P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:20:56.840344 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 971e0913f7454a4a9ec25ac3891d2b1e (table default.Table_2d9dbcf42731420199dabcdd3f6df616 [id=e6c357c8899f430b95c9c55084999d6f]) successfully deleted
W20260924 00:20:56.842900 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 1877e10d1db845f2a001063fb3bffef4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1877e10d1db845f2a001063fb3bffef4 already in progress: creating tablet
I20260924 00:20:56.822235 10632 tablet_service.cc:1514] Processing DeleteTablet for tablet a1a79421de4049aea258261ff9bd00b4 with delete_type TABLET_DATA_DELETED (replaced by tablet 1ec36d59b5114feea144bcb72794136c) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:56.844296 11049 tablet_replica.cc:330] T a1a79421de4049aea258261ff9bd00b4 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
W20260924 00:20:56.845278 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet b87bc7d0b65e4153aa649444fcc2c128 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b87bc7d0b65e4153aa649444fcc2c128 already in progress: creating tablet
I20260924 00:20:56.846437 10633 tablet_service.cc:1467] Processing CreateTablet for tablet e0074b1184ec4883ab1016330d98c968 (DEFAULT_TABLE table=default.Table_a54d4fd8353e42a4a00cb4e5c689da7b [id=485c42c942b74abbb32bea36d25bc184]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:56.847457 10633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0074b1184ec4883ab1016330d98c968. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:56.850497 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0074b1184ec4883ab1016330d98c968. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:56.853776 10635 tablet_service.cc:1467] Processing CreateTablet for tablet ccb26e0fbc1948bc9a00e9e26c4d1139 (DEFAULT_TABLE table=default.Table_d39fffcfdad94ae080bcf81a608b2424 [id=7e9e663acedf4a1fae2f04cde97b7907]), partition=RANGE (key) PARTITION UNBOUNDED
W20260924 00:20:56.854066 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 5f84857aad964b01b94e69af13007a55 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5f84857aad964b01b94e69af13007a55 already in progress: creating tablet
I20260924 00:20:56.856607 10636 tablet_service.cc:1467] Processing CreateTablet for tablet c212525586544d8f9d92bc4b8bd8acaa (DEFAULT_TABLE table=default.Table_ef16ff2bfc9b42219f0b6d5f957312f6 [id=fbdec8eccace4eb3864a6cdd95795ce3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:56.857625 10636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c212525586544d8f9d92bc4b8bd8acaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:56.820891 11046 raft_consensus.cc:2726] T a1a79421de4049aea258261ff9bd00b4 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:20:56.859390 11040 tablet_bootstrap.cc:654] T 720e1a36bcfb4260b3689d99a7358205 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:56.863147 10635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccb26e0fbc1948bc9a00e9e26c4d1139. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:56.864217 10369 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet ac559183fbef4e21b190a9a61a50e1e1
I20260924 00:20:56.864619 10372 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_3fb6b1e6699146b0aed9c1e5fa8b16f0" } new_table_name: "default.Table_d61af65b53ce4871b7b24752281af745" modify_external_catalogs: true
I20260924 00:20:56.867457 10632 tablet_service.cc:1514] Processing DeleteTablet for tablet ac559183fbef4e21b190a9a61a50e1e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:56 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:56.868227 11049 raft_consensus.cc:2229] T a1a79421de4049aea258261ff9bd00b4 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:56.868415 10776 tablet_service.cc:1514] Processing DeleteTablet for tablet ac559183fbef4e21b190a9a61a50e1e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:56 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:20:56.870244 11049 raft_consensus.cc:2258] T a1a79421de4049aea258261ff9bd00b4 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:56.870990 10921 tablet_service.cc:1514] Processing DeleteTablet for tablet ac559183fbef4e21b190a9a61a50e1e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:56 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:20:56.872272 11049 ts_tablet_manager.cc:1905] T a1a79421de4049aea258261ff9bd00b4 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:56.875801 10370 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_6ec498f485f04dd8a9110da631ea2824" } new_table_name: "default.Table_37422ca9fa2a428fbe4cf3c4fa033ca9" modify_external_catalogs: true
I20260924 00:20:56.887609 10369 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_451536c1b7444188b1dcab7ce70d663f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:20:56.888907 11041 tablet_bootstrap.cc:492] T 720e1a36bcfb4260b3689d99a7358205 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
W20260924 00:20:56.889199 10369 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_451536c1b7444188b1dcab7ce70d663f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:56.889400 11041 ts_tablet_manager.cc:1397] T 720e1a36bcfb4260b3689d99a7358205 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.141s user 0.013s sys 0.002s
I20260924 00:20:56.900041 10776 tablet_service.cc:1514] Processing DeleteTablet for tablet 971e0913f7454a4a9ec25ac3891d2b1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:55 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:20:56.899706 10374 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_9abf361701cd4e6faee21d3e41dfeae4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:20:56.891710 10373 catalog_manager.cc:6040] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: default.Table_8d75520690044effa0a919d3715e4a2f [id=30a3f7c8fb9c46cf8b883eedae25d938] alter complete (version 3)
W20260924 00:20:56.901480 10374 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9abf361701cd4e6faee21d3e41dfeae4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:56.906288 11041 raft_consensus.cc:348] T 720e1a36bcfb4260b3689d99a7358205 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:56.861668 10916 tablet_service.cc:1467] Processing CreateTablet for tablet c212525586544d8f9d92bc4b8bd8acaa (DEFAULT_TABLE table=default.Table_ef16ff2bfc9b42219f0b6d5f957312f6 [id=fbdec8eccace4eb3864a6cdd95795ce3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:56.909407 10916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c212525586544d8f9d92bc4b8bd8acaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:56.920452 11041 raft_consensus.cc:374] T 720e1a36bcfb4260b3689d99a7358205 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:56.920816 11041 raft_consensus.cc:729] T 720e1a36bcfb4260b3689d99a7358205 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:20:56.925482 10941 raft_consensus.cc:3037] T 1e5beeac2a1d4cb18b5d3d2408fd574b P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:20:56.926438 11051 tablet_replica.cc:330] T 8fc17bdba09b40b7a3280c64121ff508 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:20:56.927124 11051 raft_consensus.cc:2229] T 8fc17bdba09b40b7a3280c64121ff508 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:56.926065 11041 consensus_queue.cc:260] T 720e1a36bcfb4260b3689d99a7358205 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:56.927631 11051 raft_consensus.cc:2258] T 8fc17bdba09b40b7a3280c64121ff508 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:56.930382 11051 ts_tablet_manager.cc:1905] T 8fc17bdba09b40b7a3280c64121ff508 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:56.935117 11041 ts_tablet_manager.cc:1428] T 720e1a36bcfb4260b3689d99a7358205 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.045s user 0.003s sys 0.001s
W20260924 00:20:56.938220 10372 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ae0c9b785195491987dd8fe397d704e0 has no consensus state
W20260924 00:20:56.939790 10370 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet eeb11092b803467e9db732468350e852 has no consensus state
I20260924 00:20:56.951130 11041 tablet_bootstrap.cc:492] T 9a288e9b6f804283b6d48e4d4e1dd828 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:20:56.955310 11050 ts_tablet_manager.cc:1918] T e1864ca7a2e44611b6bb4e87f04dc61f P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:56.959481 10632 tablet_service.cc:1514] Processing DeleteTablet for tablet 720e1a36bcfb4260b3689d99a7358205 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:56 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:56.957734 11050 log.cc:1196] T e1864ca7a2e44611b6bb4e87f04dc61f P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/e1864ca7a2e44611b6bb4e87f04dc61f
I20260924 00:20:56.965945 11050 ts_tablet_manager.cc:1939] T e1864ca7a2e44611b6bb4e87f04dc61f P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:20:56.969851 11040 tablet_bootstrap.cc:492] T 720e1a36bcfb4260b3689d99a7358205 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:20:56.970235 11040 ts_tablet_manager.cc:1397] T 720e1a36bcfb4260b3689d99a7358205 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.134s user 0.006s sys 0.003s
I20260924 00:20:56.971167 11097 tablet.cc:1717] T 1e5beeac2a1d4cb18b5d3d2408fd574b P b10a9082a37b46f09d581944914c1670: Skipping requested alter to schema version 3, tablet already version 3
I20260924 00:20:56.972584 11098 tablet.cc:1717] T 1e5beeac2a1d4cb18b5d3d2408fd574b P 48fbd603abe146c585d15196e1b5be08: Skipping requested alter to schema version 3, tablet already version 3
I20260924 00:20:56.972271 11040 raft_consensus.cc:348] T 720e1a36bcfb4260b3689d99a7358205 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:56.978641 11040 raft_consensus.cc:374] T 720e1a36bcfb4260b3689d99a7358205 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:56.978962 11040 raft_consensus.cc:729] T 720e1a36bcfb4260b3689d99a7358205 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:20:56.979584 11040 consensus_queue.cc:260] T 720e1a36bcfb4260b3689d99a7358205 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:56.982051 11041 tablet_bootstrap.cc:654] T 9a288e9b6f804283b6d48e4d4e1dd828 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:56.993271 11098 tablet.cc:1717] T 1e5beeac2a1d4cb18b5d3d2408fd574b P 48fbd603abe146c585d15196e1b5be08: Skipping requested alter to schema version 3, tablet already version 3
I20260924 00:20:56.994918 11089 mvcc.cc:205] Tried to move back new op lower bound from 7332697116580593664 to 7332697112560857088. Current Snapshot: MvccSnapshot[applied={T|T < 7332697116580593664}]
I20260924 00:20:56.997593 11039 tablet_bootstrap.cc:492] T ad256de63e0948928a51d9231152ac2d P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:20:56.997532 10778 tablet_service.cc:1514] Processing DeleteTablet for tablet 720e1a36bcfb4260b3689d99a7358205 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:56 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:20:56.997992 11039 ts_tablet_manager.cc:1397] T ad256de63e0948928a51d9231152ac2d P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.258s user 0.007s sys 0.001s
I20260924 00:20:56.999382 11051 ts_tablet_manager.cc:1918] T 8fc17bdba09b40b7a3280c64121ff508 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:57.003453 11039 raft_consensus.cc:348] T ad256de63e0948928a51d9231152ac2d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:57.004155 11039 raft_consensus.cc:374] T ad256de63e0948928a51d9231152ac2d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:57.004388 11039 raft_consensus.cc:729] T ad256de63e0948928a51d9231152ac2d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:20:57.005309 11040 ts_tablet_manager.cc:1428] T 720e1a36bcfb4260b3689d99a7358205 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.035s user 0.006s sys 0.001s
I20260924 00:20:57.005039 11039 consensus_queue.cc:260] T ad256de63e0948928a51d9231152ac2d P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:56.984318 11046 raft_consensus.cc:482] T 720e1a36bcfb4260b3689d99a7358205 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:20:56.987946 11097 tablet.cc:1717] T 1e5beeac2a1d4cb18b5d3d2408fd574b P b10a9082a37b46f09d581944914c1670: Skipping requested alter to schema version 3, tablet already version 3
I20260924 00:20:57.005887 11046 raft_consensus.cc:504] T 720e1a36bcfb4260b3689d99a7358205 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:57.010021 11046 leader_election.cc:288] T 720e1a36bcfb4260b3689d99a7358205 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:20:57.014015 10632 tablet_service.cc:1467] Processing CreateTablet for tablet ae0c9b785195491987dd8fe397d704e0 (DEFAULT_TABLE table=default.Table_d61af65b53ce4871b7b24752281af745 [id=0edc6c93a1ba4c8d81755129301c83de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:57.015460 10632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae0c9b785195491987dd8fe397d704e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:57.032485 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet e1864ca7a2e44611b6bb4e87f04dc61f (table default.Table_b1ae3ebb4b3e4c43a1a684b765cfdd38 [id=9d8bc2174e884f97b364cc1d2e5af3b4]) successfully deleted
W20260924 00:20:57.033308 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 5f84857aad964b01b94e69af13007a55 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5f84857aad964b01b94e69af13007a55 already in progress: creating tablet
W20260924 00:20:57.033941 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet b87bc7d0b65e4153aa649444fcc2c128 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b87bc7d0b65e4153aa649444fcc2c128 already in progress: creating tablet
W20260924 00:20:57.034518 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet a1a79421de4049aea258261ff9bd00b4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a1a79421de4049aea258261ff9bd00b4 already in progress: creating tablet
I20260924 00:20:57.034854 10372 master_service.cc:946] ReplaceTablet: received request to replace tablet b87bc7d0b65e4153aa649444fcc2c128 from {username='slave'} at 127.0.0.1:47022
W20260924 00:20:57.035079 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 1877e10d1db845f2a001063fb3bffef4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1877e10d1db845f2a001063fb3bffef4 already in progress: creating tablet
W20260924 00:20:57.035653 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet ac559183fbef4e21b190a9a61a50e1e1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ac559183fbef4e21b190a9a61a50e1e1 already in progress: creating tablet
I20260924 00:20:57.012007 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "720e1a36bcfb4260b3689d99a7358205" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:20:57.012195 10920 tablet_service.cc:1467] Processing CreateTablet for tablet ae0c9b785195491987dd8fe397d704e0 (DEFAULT_TABLE table=default.Table_d61af65b53ce4871b7b24752281af745 [id=0edc6c93a1ba4c8d81755129301c83de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:57.038358 10920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae0c9b785195491987dd8fe397d704e0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:20:57.040198 10589 leader_election.cc:341] T 720e1a36bcfb4260b3689d99a7358205 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:20:57.026093 11051 log.cc:1196] T 8fc17bdba09b40b7a3280c64121ff508 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/8fc17bdba09b40b7a3280c64121ff508
I20260924 00:20:57.041111 10372 master_service.cc:946] ReplaceTablet: received request to replace tablet 9a288e9b6f804283b6d48e4d4e1dd828 from {username='slave'} at 127.0.0.1:47022
I20260924 00:20:57.041702 11051 ts_tablet_manager.cc:1939] T 8fc17bdba09b40b7a3280c64121ff508 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:20:57.043865 10634 tablet_service.cc:1467] Processing CreateTablet for tablet af2f58e6757a4b738a8ec38b2e228f9f (DEFAULT_TABLE table=default.Table_391ed8efc7a54515837c1b79be85d64c [id=f72ddb5979044cda93de75fcb1983ca4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:57.045234 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 8fc17bdba09b40b7a3280c64121ff508 (table default.Table_bf986279c4534aa9bf4f06adb96f16ea [id=43419db539614db8802a84dff868f6b9]) successfully deleted
I20260924 00:20:57.049347 10373 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_2d4af2cf18544b6e94ef78ac7dd3bcce" } modify_external_catalogs: true
I20260924 00:20:57.049906 10373 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_2d4af2cf18544b6e94ef78ac7dd3bcce" } modify_external_catalogs: true
W20260924 00:20:57.051108 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet a1a79421de4049aea258261ff9bd00b4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a1a79421de4049aea258261ff9bd00b4 already in progress: creating tablet
I20260924 00:20:57.008920 11039 ts_tablet_manager.cc:1428] T ad256de63e0948928a51d9231152ac2d P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
W20260924 00:20:57.053346 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 1877e10d1db845f2a001063fb3bffef4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1877e10d1db845f2a001063fb3bffef4 already in progress: creating tablet
I20260924 00:20:57.008919 10778 tablet_service.cc:1467] Processing CreateTablet for tablet af2f58e6757a4b738a8ec38b2e228f9f (DEFAULT_TABLE table=default.Table_391ed8efc7a54515837c1b79be85d64c [id=f72ddb5979044cda93de75fcb1983ca4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:57.053886 11039 tablet_bootstrap.cc:492] T 746fb7b4571f428a80495f6e9c262a7f P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:20:57.054463 10778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af2f58e6757a4b738a8ec38b2e228f9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:57.056829 10634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af2f58e6757a4b738a8ec38b2e228f9f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:20:57.057622 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet ac559183fbef4e21b190a9a61a50e1e1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ac559183fbef4e21b190a9a61a50e1e1 already in progress: creating tablet
I20260924 00:20:57.061256 10918 tablet_service.cc:1467] Processing CreateTablet for tablet af2f58e6757a4b738a8ec38b2e228f9f (DEFAULT_TABLE table=default.Table_391ed8efc7a54515837c1b79be85d64c [id=f72ddb5979044cda93de75fcb1983ca4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:57.062391 10918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af2f58e6757a4b738a8ec38b2e228f9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:57.012821 10800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "720e1a36bcfb4260b3689d99a7358205" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:20:57.062963 10800 raft_consensus.cc:2454] T 720e1a36bcfb4260b3689d99a7358205 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b10a9082a37b46f09d581944914c1670 in term 0.
I20260924 00:20:57.064096 10592 leader_election.cc:302] T 720e1a36bcfb4260b3689d99a7358205 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670; no voters: c8626e300d0644b0908797ad61bd7cef
I20260924 00:20:57.069036 10369 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_f1d482834246412aab6e26a93c7fe32c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:20:57.070327 11050 tablet_replica.cc:330] T 971e0913f7454a4a9ec25ac3891d2b1e P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:20:57.071060 11050 raft_consensus.cc:2229] T 971e0913f7454a4a9ec25ac3891d2b1e P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:57.071496 11050 raft_consensus.cc:2258] T 971e0913f7454a4a9ec25ac3891d2b1e P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:57.072101 10374 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_391ed8efc7a54515837c1b79be85d64c" } modify_external_catalogs: true
I20260924 00:20:57.072583 10374 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_391ed8efc7a54515837c1b79be85d64c" } modify_external_catalogs: true
I20260924 00:20:57.073189 11046 raft_consensus.cc:2781] T 720e1a36bcfb4260b3689d99a7358205 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:20:57.073508 11046 raft_consensus.cc:482] T 720e1a36bcfb4260b3689d99a7358205 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:20:57.073793 11046 raft_consensus.cc:3037] T 720e1a36bcfb4260b3689d99a7358205 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
W20260924 00:20:57.071221 10369 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f1d482834246412aab6e26a93c7fe32c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:57.075354 10372 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_1f2d71bd44cd479f9e0cca040494140f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:20:57.079052 10372 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1f2d71bd44cd479f9e0cca040494140f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:57.041750 10775 tablet_service.cc:1467] Processing CreateTablet for tablet ae0c9b785195491987dd8fe397d704e0 (DEFAULT_TABLE table=default.Table_d61af65b53ce4871b7b24752281af745 [id=0edc6c93a1ba4c8d81755129301c83de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:57.081852 10775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae0c9b785195491987dd8fe397d704e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:57.081862 11039 tablet_bootstrap.cc:654] T 746fb7b4571f428a80495f6e9c262a7f P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:57.085567 11046 raft_consensus.cc:504] T 720e1a36bcfb4260b3689d99a7358205 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:57.088274 10800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "720e1a36bcfb4260b3689d99a7358205" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:20:57.088822 10800 raft_consensus.cc:3037] T 720e1a36bcfb4260b3689d99a7358205 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:20:57.096660 10800 raft_consensus.cc:2454] T 720e1a36bcfb4260b3689d99a7358205 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b10a9082a37b46f09d581944914c1670 in term 1.
I20260924 00:20:57.097811 10592 leader_election.cc:302] T 720e1a36bcfb4260b3689d99a7358205 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670; no voters:
I20260924 00:20:57.102349 11050 ts_tablet_manager.cc:1905] T 971e0913f7454a4a9ec25ac3891d2b1e P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:57.104103 11049 ts_tablet_manager.cc:1918] T a1a79421de4049aea258261ff9bd00b4 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:57.104478 11049 log.cc:1196] T a1a79421de4049aea258261ff9bd00b4 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/a1a79421de4049aea258261ff9bd00b4
I20260924 00:20:57.105190 11049 ts_tablet_manager.cc:1939] T a1a79421de4049aea258261ff9bd00b4 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:20:57.106798 11039 tablet_bootstrap.cc:492] T 746fb7b4571f428a80495f6e9c262a7f P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:20:57.107177 11039 ts_tablet_manager.cc:1397] T 746fb7b4571f428a80495f6e9c262a7f P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.053s user 0.008s sys 0.000s
I20260924 00:20:57.120577 11105 raft_consensus.cc:2781] T 720e1a36bcfb4260b3689d99a7358205 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:20:57.119356 11040 tablet_bootstrap.cc:492] T a1a79421de4049aea258261ff9bd00b4 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:20:57.130525 11046 leader_election.cc:288] T 720e1a36bcfb4260b3689d99a7358205 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Requested vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:20:57.131798 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "720e1a36bcfb4260b3689d99a7358205" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef"
W20260924 00:20:57.133800 10589 leader_election.cc:341] T 720e1a36bcfb4260b3689d99a7358205 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:20:57.135156 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet a1a79421de4049aea258261ff9bd00b4 (table default.Table_b1726c09065e4fbaa2e6799b3ebdbc74 [id=4dbb68e0a32349ad8da2945db650584d]) successfully deleted
W20260924 00:20:57.136726 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet ac559183fbef4e21b190a9a61a50e1e1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ac559183fbef4e21b190a9a61a50e1e1 already in progress: creating tablet
I20260924 00:20:57.141619 11049 tablet_replica.cc:330] T 720e1a36bcfb4260b3689d99a7358205 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:20:57.143527 10370 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_fe46adb213e84b79beecb3d7e78707c1" } new_table_name: "default.Table_4010035427fa43afae7f802f2dadf83c" modify_external_catalogs: true
I20260924 00:20:57.144397 10367 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_1f2d71bd44cd479f9e0cca040494140f" } new_table_name: "default.Table_6a3c73b6e78a4066bb59da236b32ab81" modify_external_catalogs: true
I20260924 00:20:57.146288 10372 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 1e5beeac2a1d4cb18b5d3d2408fd574b (table default.Table_8d75520690044effa0a919d3715e4a2f [id=30a3f7c8fb9c46cf8b883eedae25d938]). Expected version 3 got 0
I20260924 00:20:57.148730 11039 raft_consensus.cc:348] T 746fb7b4571f428a80495f6e9c262a7f P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:57.149394 11039 raft_consensus.cc:374] T 746fb7b4571f428a80495f6e9c262a7f P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:57.149622 11039 raft_consensus.cc:729] T 746fb7b4571f428a80495f6e9c262a7f P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:20:57.153190 11039 consensus_queue.cc:260] T 746fb7b4571f428a80495f6e9c262a7f P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:57.157928 11039 ts_tablet_manager.cc:1428] T 746fb7b4571f428a80495f6e9c262a7f P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.051s user 0.005s sys 0.000s
I20260924 00:20:57.161124 11106 tablet.cc:1722] T 1e5beeac2a1d4cb18b5d3d2408fd574b P c8626e300d0644b0908797ad61bd7cef: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260924 00:20:57.167282 11040 tablet_bootstrap.cc:654] T a1a79421de4049aea258261ff9bd00b4 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:57.175050 11039 tablet_bootstrap.cc:492] T 9a288e9b6f804283b6d48e4d4e1dd828 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:20:57.177917 11041 tablet_bootstrap.cc:492] T 9a288e9b6f804283b6d48e4d4e1dd828 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:20:57.178529 11041 ts_tablet_manager.cc:1397] T 9a288e9b6f804283b6d48e4d4e1dd828 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.228s user 0.010s sys 0.000s
I20260924 00:20:57.181474 11041 raft_consensus.cc:348] T 9a288e9b6f804283b6d48e4d4e1dd828 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:57.182166 11041 raft_consensus.cc:374] T 9a288e9b6f804283b6d48e4d4e1dd828 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:57.182581 11041 raft_consensus.cc:729] T 9a288e9b6f804283b6d48e4d4e1dd828 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:20:57.186566 11105 raft_consensus.cc:686] T 720e1a36bcfb4260b3689d99a7358205 P b10a9082a37b46f09d581944914c1670 [term 1 LEADER]: Becoming Leader. State: Replica: b10a9082a37b46f09d581944914c1670, State: Running, Role: LEADER
I20260924 00:20:57.187402 11105 consensus_queue.cc:237] T 720e1a36bcfb4260b3689d99a7358205 P b10a9082a37b46f09d581944914c1670 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:57.183179 11041 consensus_queue.cc:260] T 9a288e9b6f804283b6d48e4d4e1dd828 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:57.213639 11041 ts_tablet_manager.cc:1428] T 9a288e9b6f804283b6d48e4d4e1dd828 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.035s user 0.006s sys 0.000s
I20260924 00:20:57.218021 11041 tablet_bootstrap.cc:492] T 5f84857aad964b01b94e69af13007a55 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:20:57.220266 11040 tablet_bootstrap.cc:492] T a1a79421de4049aea258261ff9bd00b4 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:20:57.220736 11040 ts_tablet_manager.cc:1397] T a1a79421de4049aea258261ff9bd00b4 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.102s user 0.008s sys 0.001s
I20260924 00:20:57.222901 11040 raft_consensus.cc:348] T a1a79421de4049aea258261ff9bd00b4 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
W20260924 00:20:57.225618 10367 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 86374d72238d42c29542e649a7a0200d has no consensus state
W20260924 00:20:57.226730 10370 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c83942a74816465fa7f3f2c3e18c98bb consensus state has no leader
I20260924 00:20:57.233938 11040 raft_consensus.cc:374] T a1a79421de4049aea258261ff9bd00b4 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:57.234185 11040 raft_consensus.cc:729] T a1a79421de4049aea258261ff9bd00b4 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:20:57.233692 11050 ts_tablet_manager.cc:1918] T 971e0913f7454a4a9ec25ac3891d2b1e P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:57.234995 11050 log.cc:1196] T 971e0913f7454a4a9ec25ac3891d2b1e P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/971e0913f7454a4a9ec25ac3891d2b1e
I20260924 00:20:57.235707 11050 ts_tablet_manager.cc:1939] T 971e0913f7454a4a9ec25ac3891d2b1e P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:20:57.237214 11040 consensus_queue.cc:260] T a1a79421de4049aea258261ff9bd00b4 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:57.238106 11039 tablet_bootstrap.cc:654] T 9a288e9b6f804283b6d48e4d4e1dd828 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:57.261632 10367 master_service.cc:946] ReplaceTablet: received request to replace tablet 1e5beeac2a1d4cb18b5d3d2408fd574b from {username='slave'} at 127.0.0.1:47022
I20260924 00:20:57.261426 10369 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_bc2eec8db7494414acf50558e06cd11e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:20:57.262998 10369 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bc2eec8db7494414acf50558e06cd11e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:57.266893 11106 tablet.cc:1717] T 1e5beeac2a1d4cb18b5d3d2408fd574b P c8626e300d0644b0908797ad61bd7cef: Skipping requested alter to schema version 3, tablet already version 3
I20260924 00:20:57.263222 10368 master_service.cc:946] ReplaceTablet: received request to replace tablet 9a288e9b6f804283b6d48e4d4e1dd828 from {username='slave'} at 127.0.0.1:47022
I20260924 00:20:57.267751 10372 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_321e6959ada84ccea6413114309bc3b8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:20:57.270407 10372 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_321e6959ada84ccea6413114309bc3b8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:57.280085 11040 ts_tablet_manager.cc:1428] T a1a79421de4049aea258261ff9bd00b4 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.059s user 0.003s sys 0.002s
I20260924 00:20:57.281939 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 971e0913f7454a4a9ec25ac3891d2b1e (table default.Table_2d9dbcf42731420199dabcdd3f6df616 [id=e6c357c8899f430b95c9c55084999d6f]) successfully deleted
I20260924 00:20:57.284241 11050 tablet_replica.cc:330] T 720e1a36bcfb4260b3689d99a7358205 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:20:57.289286 11039 tablet_bootstrap.cc:492] T 9a288e9b6f804283b6d48e4d4e1dd828 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:20:57.279023 11041 tablet_bootstrap.cc:654] T 5f84857aad964b01b94e69af13007a55 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:57.289672 11039 ts_tablet_manager.cc:1397] T 9a288e9b6f804283b6d48e4d4e1dd828 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.115s user 0.013s sys 0.001s
I20260924 00:20:57.291742 11039 raft_consensus.cc:348] T 9a288e9b6f804283b6d48e4d4e1dd828 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:57.292378 11039 raft_consensus.cc:374] T 9a288e9b6f804283b6d48e4d4e1dd828 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:57.293208 11113 tablet.cc:1717] T 1e5beeac2a1d4cb18b5d3d2408fd574b P c8626e300d0644b0908797ad61bd7cef: Skipping requested alter to schema version 3, tablet already version 3
I20260924 00:20:57.294075 11050 raft_consensus.cc:2229] T 720e1a36bcfb4260b3689d99a7358205 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:57.291790 11040 tablet_bootstrap.cc:492] T 746fb7b4571f428a80495f6e9c262a7f P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:20:57.296257 11050 raft_consensus.cc:2258] T 720e1a36bcfb4260b3689d99a7358205 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:57.300716 10775 tablet_service.cc:1514] Processing DeleteTablet for tablet a1a79421de4049aea258261ff9bd00b4 with delete_type TABLET_DATA_DELETED (replaced by tablet 1ec36d59b5114feea144bcb72794136c) from {username='slave'} at 127.0.0.1:35996
I20260924 00:20:57.301365 11050 ts_tablet_manager.cc:1905] T 720e1a36bcfb4260b3689d99a7358205 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:57.303925 11040 tablet_bootstrap.cc:654] T 746fb7b4571f428a80495f6e9c262a7f P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:57.292609 11039 raft_consensus.cc:729] T 9a288e9b6f804283b6d48e4d4e1dd828 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:20:57.316687 10918 tablet_service.cc:1467] Processing CreateTablet for tablet bcd8a6b92b8e492d96994195fa7f3ae5 (DEFAULT_TABLE table=default.Table_99da18207fc542fbb68972724c4d410e [id=f75c1da3bd734e92bc105c473236fa9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:57.317865 10918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcd8a6b92b8e492d96994195fa7f3ae5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:57.318293 10916 tablet_service.cc:1514] Processing DeleteTablet for tablet 746fb7b4571f428a80495f6e9c262a7f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:55 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:20:57.319026 11051 tablet_replica.cc:330] T 746fb7b4571f428a80495f6e9c262a7f P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:20:57.319695 11051 raft_consensus.cc:2229] T 746fb7b4571f428a80495f6e9c262a7f P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:57.320094 11051 raft_consensus.cc:2258] T 746fb7b4571f428a80495f6e9c262a7f P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:57.320668 11107 raft_consensus.cc:482] T ad256de63e0948928a51d9231152ac2d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:20:57.321120 11107 raft_consensus.cc:504] T ad256de63e0948928a51d9231152ac2d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:57.322106 11051 ts_tablet_manager.cc:1905] T 746fb7b4571f428a80495f6e9c262a7f P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:57.323117 11107 leader_election.cc:288] T ad256de63e0948928a51d9231152ac2d P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:20:57.324100 10941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad256de63e0948928a51d9231152ac2d" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:20:57.324793 10941 raft_consensus.cc:2454] T ad256de63e0948928a51d9231152ac2d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b10a9082a37b46f09d581944914c1670 in term 0.
I20260924 00:20:57.325805 10800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad256de63e0948928a51d9231152ac2d" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:20:57.327279 10589 leader_election.cc:302] T ad256de63e0948928a51d9231152ac2d P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
W20260924 00:20:57.327809 10592 leader_election.cc:341] T ad256de63e0948928a51d9231152ac2d P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:20:57.328616 11107 raft_consensus.cc:2781] T ad256de63e0948928a51d9231152ac2d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:20:57.328934 11107 raft_consensus.cc:482] T ad256de63e0948928a51d9231152ac2d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:20:57.329242 11107 raft_consensus.cc:3037] T ad256de63e0948928a51d9231152ac2d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:20:57.337314 11039 consensus_queue.cc:260] T 9a288e9b6f804283b6d48e4d4e1dd828 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:57.337924 11040 tablet_bootstrap.cc:492] T 746fb7b4571f428a80495f6e9c262a7f P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:20:57.342212 11040 ts_tablet_manager.cc:1397] T 746fb7b4571f428a80495f6e9c262a7f P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.053s user 0.007s sys 0.002s
I20260924 00:20:57.344657 11039 ts_tablet_manager.cc:1428] T 9a288e9b6f804283b6d48e4d4e1dd828 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.055s user 0.005s sys 0.001s
I20260924 00:20:57.346714 11040 raft_consensus.cc:348] T 746fb7b4571f428a80495f6e9c262a7f P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
W20260924 00:20:57.350006 10361 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet c83942a74816465fa7f3f2c3e18c98bb (table default.Table_4010035427fa43afae7f802f2dadf83c [id=7eb84ebbe346450eb2b944db685c414b], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet c83942a74816465fa7f3f2c3e18c98bb consensus state has no leader
I20260924 00:20:57.350920 11040 raft_consensus.cc:374] T 746fb7b4571f428a80495f6e9c262a7f P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:57.351507 11040 raft_consensus.cc:729] T 746fb7b4571f428a80495f6e9c262a7f P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:20:57.355175 10632 tablet_service.cc:1467] Processing CreateTablet for tablet bcd8a6b92b8e492d96994195fa7f3ae5 (DEFAULT_TABLE table=default.Table_99da18207fc542fbb68972724c4d410e [id=f75c1da3bd734e92bc105c473236fa9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:57.356235 10632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcd8a6b92b8e492d96994195fa7f3ae5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:57.361847 10370 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_fcb196dda8a044e8b443fa810c5b8d41"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:20:57.362715 11039 tablet_bootstrap.cc:492] T 720e1a36bcfb4260b3689d99a7358205 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
W20260924 00:20:57.364305 10370 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fcb196dda8a044e8b443fa810c5b8d41 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:57.365504 10775 tablet_service.cc:1467] Processing CreateTablet for tablet bcd8a6b92b8e492d96994195fa7f3ae5 (DEFAULT_TABLE table=default.Table_99da18207fc542fbb68972724c4d410e [id=f75c1da3bd734e92bc105c473236fa9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:57.366581 10775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcd8a6b92b8e492d96994195fa7f3ae5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:57.371703 10778 tablet_service.cc:1467] Processing CreateTablet for tablet eeb11092b803467e9db732468350e852 (DEFAULT_TABLE table=default.Table_37422ca9fa2a428fbe4cf3c4fa033ca9 [id=7a76983970cd491a8e8635db379bdd10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:57.372745 10778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eeb11092b803467e9db732468350e852. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:57.373090 10635 tablet_service.cc:1467] Processing CreateTablet for tablet eeb11092b803467e9db732468350e852 (DEFAULT_TABLE table=default.Table_37422ca9fa2a428fbe4cf3c4fa033ca9 [id=7a76983970cd491a8e8635db379bdd10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:57.384931 10635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eeb11092b803467e9db732468350e852. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:57.388986 11039 tablet_bootstrap.cc:654] T 720e1a36bcfb4260b3689d99a7358205 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:57.392742 11051 ts_tablet_manager.cc:1918] T 746fb7b4571f428a80495f6e9c262a7f P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:57.393146 11051 log.cc:1196] T 746fb7b4571f428a80495f6e9c262a7f P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/746fb7b4571f428a80495f6e9c262a7f
I20260924 00:20:57.393743 11051 ts_tablet_manager.cc:1939] T 746fb7b4571f428a80495f6e9c262a7f P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:20:57.402364 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 746fb7b4571f428a80495f6e9c262a7f (table default.Table_05ea79d978c74171980cf4ffdc8dd08e [id=96e09747aa014d5bb96e5d991b38c039]) successfully deleted
I20260924 00:20:57.405623 11050 ts_tablet_manager.cc:1918] T 720e1a36bcfb4260b3689d99a7358205 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:57.406026 11050 log.cc:1196] T 720e1a36bcfb4260b3689d99a7358205 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/720e1a36bcfb4260b3689d99a7358205
I20260924 00:20:57.406422 10373 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 6ad838c884fc4b1d8f98781e7190e3df
I20260924 00:20:57.406623 11050 ts_tablet_manager.cc:1939] T 720e1a36bcfb4260b3689d99a7358205 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:20:57.407985 10916 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ad838c884fc4b1d8f98781e7190e3df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:57 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:20:57.354164 11040 consensus_queue.cc:260] T 746fb7b4571f428a80495f6e9c262a7f P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
W20260924 00:20:57.409281 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 6ad838c884fc4b1d8f98781e7190e3df because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6ad838c884fc4b1d8f98781e7190e3df already in progress: creating tablet
I20260924 00:20:57.409767 10774 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ad838c884fc4b1d8f98781e7190e3df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:57 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:20:57.411535 10633 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ad838c884fc4b1d8f98781e7190e3df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:57 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:57.414646 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 720e1a36bcfb4260b3689d99a7358205 (table default.Table_b1ae3ebb4b3e4c43a1a684b765cfdd38 [id=9d8bc2174e884f97b364cc1d2e5af3b4]) successfully deleted
I20260924 00:20:57.414870 11049 raft_consensus.cc:2229] T 720e1a36bcfb4260b3689d99a7358205 P b10a9082a37b46f09d581944914c1670 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:20:57.415967 11049 pending_rounds.cc:62] T 720e1a36bcfb4260b3689d99a7358205 P b10a9082a37b46f09d581944914c1670: Trying to abort 1 pending ops.
I20260924 00:20:57.416177 11049 pending_rounds.cc:68] T 720e1a36bcfb4260b3689d99a7358205 P b10a9082a37b46f09d581944914c1670: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332697117455724544 op_type: NO_OP noop_request { }
I20260924 00:20:57.416551 11049 raft_consensus.cc:2866] T 720e1a36bcfb4260b3689d99a7358205 P b10a9082a37b46f09d581944914c1670 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260924 00:20:57.416486 10774 tablet_service.cc:1467] Processing CreateTablet for tablet 1ec36d59b5114feea144bcb72794136c (DEFAULT_TABLE table=default.Table_b1726c09065e4fbaa2e6799b3ebdbc74 [id=4dbb68e0a32349ad8da2945db650584d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:57.417487 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ec36d59b5114feea144bcb72794136c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:57.417398 10633 tablet_service.cc:1467] Processing CreateTablet for tablet 1ec36d59b5114feea144bcb72794136c (DEFAULT_TABLE table=default.Table_b1726c09065e4fbaa2e6799b3ebdbc74 [id=4dbb68e0a32349ad8da2945db650584d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:57.418341 10633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ec36d59b5114feea144bcb72794136c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:57.424036 10916 tablet_service.cc:1467] Processing CreateTablet for tablet eeb11092b803467e9db732468350e852 (DEFAULT_TABLE table=default.Table_37422ca9fa2a428fbe4cf3c4fa033ca9 [id=7a76983970cd491a8e8635db379bdd10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:57.425022 10916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eeb11092b803467e9db732468350e852. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:57.416829 11049 raft_consensus.cc:2258] T 720e1a36bcfb4260b3689d99a7358205 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:57.428781 10920 tablet_service.cc:1467] Processing CreateTablet for tablet 1ec36d59b5114feea144bcb72794136c (DEFAULT_TABLE table=default.Table_b1726c09065e4fbaa2e6799b3ebdbc74 [id=4dbb68e0a32349ad8da2945db650584d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:57.429948 10920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ec36d59b5114feea144bcb72794136c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:57.433463 11049 ts_tablet_manager.cc:1905] T 720e1a36bcfb4260b3689d99a7358205 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:57.438665 10780 tablet_service.cc:1514] Processing DeleteTablet for tablet 746fb7b4571f428a80495f6e9c262a7f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:55 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:20:57.441375 10371 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_8d75520690044effa0a919d3715e4a2f" } new_table_name: "default.Table_a5d798f7819e403a96f27e98cb0af2bb" modify_external_catalogs: true
I20260924 00:20:57.441686 10366 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_c879109e72984beb85e10878ed02c18f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:20:57.443845 10366 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c879109e72984beb85e10878ed02c18f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:57.446755 10372 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_37422ca9fa2a428fbe4cf3c4fa033ca9" } new_table_name: "default.Table_0e5a147af0824c1d888bee1c066cb368" modify_external_catalogs: true
I20260924 00:20:57.454399 10636 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a288e9b6f804283b6d48e4d4e1dd828 with delete_type TABLET_DATA_DELETED (replaced by tablet 1877e10d1db845f2a001063fb3bffef4) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:57.377087 11107 raft_consensus.cc:504] T ad256de63e0948928a51d9231152ac2d P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:57.459437 10800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad256de63e0948928a51d9231152ac2d" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
W20260924 00:20:57.460563 10592 leader_election.cc:341] T ad256de63e0948928a51d9231152ac2d P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:20:57.461666 11040 ts_tablet_manager.cc:1428] T 746fb7b4571f428a80495f6e9c262a7f P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.119s user 0.003s sys 0.002s
I20260924 00:20:57.465337 11107 leader_election.cc:288] T ad256de63e0948928a51d9231152ac2d P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Requested vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:20:57.466789 10941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad256de63e0948928a51d9231152ac2d" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef"
I20260924 00:20:57.467360 10941 raft_consensus.cc:3037] T ad256de63e0948928a51d9231152ac2d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:20:57.474954 11039 tablet_bootstrap.cc:492] T 720e1a36bcfb4260b3689d99a7358205 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:20:57.475320 11039 ts_tablet_manager.cc:1397] T 720e1a36bcfb4260b3689d99a7358205 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.113s user 0.004s sys 0.005s
I20260924 00:20:57.475452 10941 raft_consensus.cc:2454] T ad256de63e0948928a51d9231152ac2d P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b10a9082a37b46f09d581944914c1670 in term 1.
I20260924 00:20:57.477419 11039 raft_consensus.cc:348] T 720e1a36bcfb4260b3689d99a7358205 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:57.478111 11039 raft_consensus.cc:374] T 720e1a36bcfb4260b3689d99a7358205 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:57.478334 11039 raft_consensus.cc:729] T 720e1a36bcfb4260b3689d99a7358205 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
W20260924 00:20:57.478341 11071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:20:57.479189 11071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
I20260924 00:20:57.478976 11039 consensus_queue.cc:260] T 720e1a36bcfb4260b3689d99a7358205 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
W20260924 00:20:57.479874 11071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260924 00:20:57.484145 11040 tablet_bootstrap.cc:492] T ad256de63e0948928a51d9231152ac2d P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:20:57.504809 11050 tablet_replica.cc:330] T a1a79421de4049aea258261ff9bd00b4 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:20:57.505489 11050 raft_consensus.cc:2229] T a1a79421de4049aea258261ff9bd00b4 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:57.505875 11050 raft_consensus.cc:2258] T a1a79421de4049aea258261ff9bd00b4 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:57.505661 10374 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet af2f58e6757a4b738a8ec38b2e228f9f
I20260924 00:20:57.508512 10780 tablet_service.cc:1514] Processing DeleteTablet for tablet af2f58e6757a4b738a8ec38b2e228f9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:57 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:20:57.508142 10589 leader_election.cc:302] T ad256de63e0948928a51d9231152ac2d P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08
I20260924 00:20:57.510768 10917 tablet_service.cc:1514] Processing DeleteTablet for tablet af2f58e6757a4b738a8ec38b2e228f9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:57 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:20:57.512430 10367 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 1e5beeac2a1d4cb18b5d3d2408fd574b
I20260924 00:20:57.513408 10368 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_5cf772cb831f4e7e97430617b962db80"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:20:57.513833 10367 catalog_manager.cc:6565] ReplaceTablet: tablet 1e5beeac2a1d4cb18b5d3d2408fd574b deleted and replaced by tablet 5b0345252c3c496db2987c30adce3da7
I20260924 00:20:57.514354 10780 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e5beeac2a1d4cb18b5d3d2408fd574b with delete_type TABLET_DATA_DELETED (replaced by tablet 5b0345252c3c496db2987c30adce3da7) from {username='slave'} at 127.0.0.1:35996
I20260924 00:20:57.515820 10917 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e5beeac2a1d4cb18b5d3d2408fd574b with delete_type TABLET_DATA_DELETED (replaced by tablet 5b0345252c3c496db2987c30adce3da7) from {username='slave'} at 127.0.0.1:35358
I20260924 00:20:57.515848 10636 tablet_service.cc:1514] Processing DeleteTablet for tablet af2f58e6757a4b738a8ec38b2e228f9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:57 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:57.516892 10636 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e5beeac2a1d4cb18b5d3d2408fd574b with delete_type TABLET_DATA_DELETED (replaced by tablet 5b0345252c3c496db2987c30adce3da7) from {username='slave'} at 127.0.0.1:46896
W20260924 00:20:57.518286 10368 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5cf772cb831f4e7e97430617b962db80 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:57.521003 11051 tablet_replica.cc:330] T 1e5beeac2a1d4cb18b5d3d2408fd574b P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:20:57.521678 11051 raft_consensus.cc:2229] T 1e5beeac2a1d4cb18b5d3d2408fd574b P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:57.522253 11051 raft_consensus.cc:2258] T 1e5beeac2a1d4cb18b5d3d2408fd574b P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 00:20:57.524160 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet af2f58e6757a4b738a8ec38b2e228f9f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet af2f58e6757a4b738a8ec38b2e228f9f already in progress: creating tablet
I20260924 00:20:57.524490 11051 ts_tablet_manager.cc:1905] T 1e5beeac2a1d4cb18b5d3d2408fd574b P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:57.524976 10367 master_service.cc:946] ReplaceTablet: received request to replace tablet 453ce17089ce44ba8da67a0893301c3c from {username='slave'} at 127.0.0.1:47022
I20260924 00:20:57.525810 11107 raft_consensus.cc:2781] T ad256de63e0948928a51d9231152ac2d P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:20:57.526345 11107 raft_consensus.cc:686] T ad256de63e0948928a51d9231152ac2d P b10a9082a37b46f09d581944914c1670 [term 1 LEADER]: Becoming Leader. State: Replica: b10a9082a37b46f09d581944914c1670, State: Running, Role: LEADER
I20260924 00:20:57.528132 11107 consensus_queue.cc:237] T ad256de63e0948928a51d9231152ac2d P b10a9082a37b46f09d581944914c1670 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:57.533564 11041 tablet_bootstrap.cc:492] T 5f84857aad964b01b94e69af13007a55 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:20:57.536020 11041 ts_tablet_manager.cc:1397] T 5f84857aad964b01b94e69af13007a55 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.318s user 0.009s sys 0.012s
W20260924 00:20:57.536180 10361 consensus_peers.cc:487] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 -> Peer f8bd0b3184f84f5f8da10c9c21472f92 (127.10.4.124:35345): Couldn't send request to peer f8bd0b3184f84f5f8da10c9c21472f92. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.124:35345: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 00:20:57.536321 11071 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:20:57.536667 11071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:20:57.537022 11071 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:20:57.537350 11071 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:20:57.537501 11050 ts_tablet_manager.cc:1905] T a1a79421de4049aea258261ff9bd00b4 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:57.537595 11039 ts_tablet_manager.cc:1428] T 720e1a36bcfb4260b3689d99a7358205 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.062s user 0.006s sys 0.000s
I20260924 00:20:57.539847 11041 raft_consensus.cc:348] T 5f84857aad964b01b94e69af13007a55 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:57.544088 11039 tablet_bootstrap.cc:492] T a1a79421de4049aea258261ff9bd00b4 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:20:57.550565 11041 raft_consensus.cc:374] T 5f84857aad964b01b94e69af13007a55 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:57.550925 11041 raft_consensus.cc:729] T 5f84857aad964b01b94e69af13007a55 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:20:57.551756 11041 consensus_queue.cc:260] T 5f84857aad964b01b94e69af13007a55 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:57.555888 11041 ts_tablet_manager.cc:1428] T 5f84857aad964b01b94e69af13007a55 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.018s user 0.004s sys 0.001s
I20260924 00:20:57.557569 11041 tablet_bootstrap.cc:492] T b87bc7d0b65e4153aa649444fcc2c128 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:20:57.573096 11039 tablet_bootstrap.cc:654] T a1a79421de4049aea258261ff9bd00b4 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:57.577087 11041 tablet_bootstrap.cc:654] T b87bc7d0b65e4153aa649444fcc2c128 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:57.580314 10373 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_f21f079a72ef4c6ca5376a661c2906e8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:20:57.581940 10373 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f21f079a72ef4c6ca5376a661c2906e8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:57.583159 11051 ts_tablet_manager.cc:1918] T 1e5beeac2a1d4cb18b5d3d2408fd574b P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20260924 00:20:57.583478 11051 log.cc:1196] T 1e5beeac2a1d4cb18b5d3d2408fd574b P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/1e5beeac2a1d4cb18b5d3d2408fd574b
I20260924 00:20:57.584059 11051 ts_tablet_manager.cc:1939] T 1e5beeac2a1d4cb18b5d3d2408fd574b P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
W20260924 00:20:57.593206 10371 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5b0345252c3c496db2987c30adce3da7 has no consensus state
I20260924 00:20:57.594403 11039 tablet_bootstrap.cc:492] T a1a79421de4049aea258261ff9bd00b4 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:20:57.594741 11039 ts_tablet_manager.cc:1397] T a1a79421de4049aea258261ff9bd00b4 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.051s user 0.008s sys 0.000s
I20260924 00:20:57.598826 11071 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.4.84:38335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-2/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.10.4.126:34479,127.10.4.125:34487,127.10.4.124:35345
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.4.124:35345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.4.124
--webserver_port=35831
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:32:10 UTC on ccycloud.quasar-wfxeao.root.comops.site
build id 4668917
TSAN enabled
I20260924 00:20:57.600281 11071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:20:57.602825 11071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 00:20:57.603852 11039 raft_consensus.cc:348] T a1a79421de4049aea258261ff9bd00b4 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:57.604485 11039 raft_consensus.cc:374] T a1a79421de4049aea258261ff9bd00b4 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:57.604718 11039 raft_consensus.cc:729] T a1a79421de4049aea258261ff9bd00b4 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:20:57.605392 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 1e5beeac2a1d4cb18b5d3d2408fd574b (table default.Table_a5d798f7819e403a96f27e98cb0af2bb [id=30a3f7c8fb9c46cf8b883eedae25d938]) successfully deleted
I20260924 00:20:57.605379 11039 consensus_queue.cc:260] T a1a79421de4049aea258261ff9bd00b4 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
W20260924 00:20:57.612457 10372 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet eeb11092b803467e9db732468350e852 consensus state has no leader
I20260924 00:20:57.617303 10918 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a288e9b6f804283b6d48e4d4e1dd828 with delete_type TABLET_DATA_DELETED (replaced by tablet 1877e10d1db845f2a001063fb3bffef4) from {username='slave'} at 127.0.0.1:35358
I20260924 00:20:57.617650 10372 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_ef16ff2bfc9b42219f0b6d5f957312f6" } modify_external_catalogs: true
I20260924 00:20:57.618198 10372 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_ef16ff2bfc9b42219f0b6d5f957312f6" } modify_external_catalogs: true
I20260924 00:20:57.619059 11051 tablet_replica.cc:330] T 9a288e9b6f804283b6d48e4d4e1dd828 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:20:57.620689 11039 ts_tablet_manager.cc:1428] T a1a79421de4049aea258261ff9bd00b4 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.026s user 0.007s sys 0.000s
I20260924 00:20:57.621537 11051 raft_consensus.cc:2229] T 9a288e9b6f804283b6d48e4d4e1dd828 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:57.621977 11051 raft_consensus.cc:2258] T 9a288e9b6f804283b6d48e4d4e1dd828 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:57.626144 10918 tablet_service.cc:1514] Processing DeleteTablet for tablet 720e1a36bcfb4260b3689d99a7358205 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:56 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:20:57.627396 11039 tablet_bootstrap.cc:492] T 5f84857aad964b01b94e69af13007a55 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:20:57.627622 11040 tablet_bootstrap.cc:654] T ad256de63e0948928a51d9231152ac2d P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:57.630991 11051 ts_tablet_manager.cc:1905] T 9a288e9b6f804283b6d48e4d4e1dd828 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:57.632496 10918 tablet_service.cc:1514] Processing DeleteTablet for tablet a1a79421de4049aea258261ff9bd00b4 with delete_type TABLET_DATA_DELETED (replaced by tablet 1ec36d59b5114feea144bcb72794136c) from {username='slave'} at 127.0.0.1:35358
I20260924 00:20:57.648031 11040 tablet_bootstrap.cc:492] T ad256de63e0948928a51d9231152ac2d P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:20:57.648420 11040 ts_tablet_manager.cc:1397] T ad256de63e0948928a51d9231152ac2d P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.164s user 0.009s sys 0.000s
I20260924 00:20:57.586746 11049 ts_tablet_manager.cc:1918] T 720e1a36bcfb4260b3689d99a7358205 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:20:57.649732 11049 log.cc:1196] T 720e1a36bcfb4260b3689d99a7358205 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/720e1a36bcfb4260b3689d99a7358205
I20260924 00:20:57.650449 11049 ts_tablet_manager.cc:1939] T 720e1a36bcfb4260b3689d99a7358205 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:20:57.652029 11039 tablet_bootstrap.cc:654] T 5f84857aad964b01b94e69af13007a55 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:57.653689 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 720e1a36bcfb4260b3689d99a7358205 (table default.Table_b1ae3ebb4b3e4c43a1a684b765cfdd38 [id=9d8bc2174e884f97b364cc1d2e5af3b4]) successfully deleted
I20260924 00:20:57.654331 11050 ts_tablet_manager.cc:1918] T a1a79421de4049aea258261ff9bd00b4 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260924 00:20:57.654378 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 6ad838c884fc4b1d8f98781e7190e3df because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6ad838c884fc4b1d8f98781e7190e3df already in progress: creating tablet
I20260924 00:20:57.654686 11050 log.cc:1196] T a1a79421de4049aea258261ff9bd00b4 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/a1a79421de4049aea258261ff9bd00b4
I20260924 00:20:57.655201 11049 tablet_replica.cc:330] T 9a288e9b6f804283b6d48e4d4e1dd828 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:20:57.655453 11050 ts_tablet_manager.cc:1939] T a1a79421de4049aea258261ff9bd00b4 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:20:57.655856 11049 raft_consensus.cc:2229] T 9a288e9b6f804283b6d48e4d4e1dd828 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:57.656302 11049 raft_consensus.cc:2258] T 9a288e9b6f804283b6d48e4d4e1dd828 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:57.658177 11049 ts_tablet_manager.cc:1905] T 9a288e9b6f804283b6d48e4d4e1dd828 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:57.658365 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet a1a79421de4049aea258261ff9bd00b4 (table default.Table_b1726c09065e4fbaa2e6799b3ebdbc74 [id=4dbb68e0a32349ad8da2945db650584d]) successfully deleted
W20260924 00:20:57.660995 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 6ad838c884fc4b1d8f98781e7190e3df because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6ad838c884fc4b1d8f98781e7190e3df already in progress: creating tablet
I20260924 00:20:57.662382 11050 tablet_replica.cc:330] T 746fb7b4571f428a80495f6e9c262a7f P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:20:57.663493 11050 raft_consensus.cc:2229] T 746fb7b4571f428a80495f6e9c262a7f P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:57.665731 11050 raft_consensus.cc:2258] T 746fb7b4571f428a80495f6e9c262a7f P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:57.669422 11050 ts_tablet_manager.cc:1905] T 746fb7b4571f428a80495f6e9c262a7f P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:57.674799 11040 raft_consensus.cc:348] T ad256de63e0948928a51d9231152ac2d P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:57.677440 11040 raft_consensus.cc:374] T ad256de63e0948928a51d9231152ac2d P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:57.677695 11040 raft_consensus.cc:729] T ad256de63e0948928a51d9231152ac2d P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:20:57.680711 11039 tablet_bootstrap.cc:492] T 5f84857aad964b01b94e69af13007a55 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:20:57.681120 11039 ts_tablet_manager.cc:1397] T 5f84857aad964b01b94e69af13007a55 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.054s user 0.010s sys 0.000s
I20260924 00:20:57.683939 11039 raft_consensus.cc:348] T 5f84857aad964b01b94e69af13007a55 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:57.684577 11039 raft_consensus.cc:374] T 5f84857aad964b01b94e69af13007a55 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:57.694993 11051 ts_tablet_manager.cc:1918] T 9a288e9b6f804283b6d48e4d4e1dd828 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:57.695389 11051 log.cc:1196] T 9a288e9b6f804283b6d48e4d4e1dd828 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/9a288e9b6f804283b6d48e4d4e1dd828
I20260924 00:20:57.684805 11039 raft_consensus.cc:729] T 5f84857aad964b01b94e69af13007a55 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:20:57.697613 11039 consensus_queue.cc:260] T 5f84857aad964b01b94e69af13007a55 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:57.703679 11040 consensus_queue.cc:260] T ad256de63e0948928a51d9231152ac2d P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:57.713050 10918 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f84857aad964b01b94e69af13007a55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:56 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:20:57.712755 10374 catalog_manager.cc:5495] T ad256de63e0948928a51d9231152ac2d P b10a9082a37b46f09d581944914c1670 reported cstate change: term changed from 0 to 1, leader changed from <none> to b10a9082a37b46f09d581944914c1670 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "b10a9082a37b46f09d581944914c1670" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } health_report { overall_health: UNKNOWN } } }
I20260924 00:20:57.714370 10373 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_451536c1b7444188b1dcab7ce70d663f" } modify_external_catalogs: true
I20260924 00:20:57.714888 10373 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_451536c1b7444188b1dcab7ce70d663f" } modify_external_catalogs: true
I20260924 00:20:57.716979 11039 ts_tablet_manager.cc:1428] T 5f84857aad964b01b94e69af13007a55 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.036s user 0.006s sys 0.000s
I20260924 00:20:57.695956 11051 ts_tablet_manager.cc:1939] T 9a288e9b6f804283b6d48e4d4e1dd828 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:20:57.719331 11051 tablet_replica.cc:330] T 720e1a36bcfb4260b3689d99a7358205 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:20:57.719914 11051 raft_consensus.cc:2229] T 720e1a36bcfb4260b3689d99a7358205 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:57.720287 11051 raft_consensus.cc:2258] T 720e1a36bcfb4260b3689d99a7358205 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:57.724629 11039 tablet_bootstrap.cc:492] T b87bc7d0b65e4153aa649444fcc2c128 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:20:57.726507 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 9a288e9b6f804283b6d48e4d4e1dd828 (table default.Table_5f8c295237ab4e0ea184a03a9fa458a2 [id=be698d8b82734ff5940f6915ba1d4df1]) successfully deleted
I20260924 00:20:57.727497 10778 tablet_service.cc:1467] Processing CreateTablet for tablet d44932f8f08e4ea7acc05b8ec511f045 (DEFAULT_TABLE table=default.Table_321e6959ada84ccea6413114309bc3b8 [id=8092ca5df917480aba7e26a4cedf9fbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:57.728526 10778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d44932f8f08e4ea7acc05b8ec511f045. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:57.733536 11039 tablet_bootstrap.cc:654] T b87bc7d0b65e4153aa649444fcc2c128 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:57.737701 10370 master_service.cc:946] ReplaceTablet: received request to replace tablet af2f58e6757a4b738a8ec38b2e228f9f from {username='slave'} at 127.0.0.1:47022
I20260924 00:20:57.748487 11040 ts_tablet_manager.cc:1428] T ad256de63e0948928a51d9231152ac2d P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.100s user 0.002s sys 0.005s
W20260924 00:20:57.750056 11129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:20:57.756816 11051 ts_tablet_manager.cc:1905] T 720e1a36bcfb4260b3689d99a7358205 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:57.706912 11050 ts_tablet_manager.cc:1918] T 746fb7b4571f428a80495f6e9c262a7f P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:57.757704 11050 log.cc:1196] T 746fb7b4571f428a80495f6e9c262a7f P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/746fb7b4571f428a80495f6e9c262a7f
I20260924 00:20:57.759917 11050 ts_tablet_manager.cc:1939] T 746fb7b4571f428a80495f6e9c262a7f P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:20:57.763919 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 746fb7b4571f428a80495f6e9c262a7f (table default.Table_05ea79d978c74171980cf4ffdc8dd08e [id=96e09747aa014d5bb96e5d991b38c039]) successfully deleted
I20260924 00:20:57.767351 10918 tablet_service.cc:1467] Processing CreateTablet for tablet d44932f8f08e4ea7acc05b8ec511f045 (DEFAULT_TABLE table=default.Table_321e6959ada84ccea6413114309bc3b8 [id=8092ca5df917480aba7e26a4cedf9fbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:57.768528 10918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d44932f8f08e4ea7acc05b8ec511f045. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:57.770591 10775 tablet_service.cc:1467] Processing CreateTablet for tablet f294831873d548e4b9201c1bbb964096 (DEFAULT_TABLE table=default.Table_bc2eec8db7494414acf50558e06cd11e [id=e2cdad6dfe7846ebb3c14f5611cf0f18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:57.771586 10775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f294831873d548e4b9201c1bbb964096. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:57.773589 10774 tablet_service.cc:1467] Processing CreateTablet for tablet 5d7d52be4f1f4fc29b8172b9fa878f76 (DEFAULT_TABLE table=default.Table_451536c1b7444188b1dcab7ce70d663f [id=fde3781ccd3748f2acc7ac5f3f5064bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:57.774662 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d7d52be4f1f4fc29b8172b9fa878f76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:57.776244 10780 tablet_service.cc:1467] Processing CreateTablet for tablet 96960f3061fb43e9939b470f49edf1e9 (DEFAULT_TABLE table=default.Table_9abf361701cd4e6faee21d3e41dfeae4 [id=6251651eb5944ee086dd564d86204148]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:57.777201 10780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96960f3061fb43e9939b470f49edf1e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:57.779295 10916 tablet_service.cc:1467] Processing CreateTablet for tablet f294831873d548e4b9201c1bbb964096 (DEFAULT_TABLE table=default.Table_bc2eec8db7494414acf50558e06cd11e [id=e2cdad6dfe7846ebb3c14f5611cf0f18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:57.780349 10916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f294831873d548e4b9201c1bbb964096. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:57.781632 10920 tablet_service.cc:1467] Processing CreateTablet for tablet 5d7d52be4f1f4fc29b8172b9fa878f76 (DEFAULT_TABLE table=default.Table_451536c1b7444188b1dcab7ce70d663f [id=fde3781ccd3748f2acc7ac5f3f5064bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:57.782568 10920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d7d52be4f1f4fc29b8172b9fa878f76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:57.789409 11040 tablet_bootstrap.cc:492] T 5f84857aad964b01b94e69af13007a55 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:20:57.794106 11040 tablet_bootstrap.cc:654] T 5f84857aad964b01b94e69af13007a55 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:57.797501 10368 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_87fc5b8981644a5d862af47154aaa22a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:20:57.799305 10368 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_87fc5b8981644a5d862af47154aaa22a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:57.803841 10370 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_d39fffcfdad94ae080bcf81a608b2424" } new_table_name: "default.Table_01b5209b9d5c4115b0bf6aef48b51bd8" modify_external_catalogs: true
I20260924 00:20:57.804176 11039 tablet_bootstrap.cc:492] T b87bc7d0b65e4153aa649444fcc2c128 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:20:57.804577 11039 ts_tablet_manager.cc:1397] T b87bc7d0b65e4153aa649444fcc2c128 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.080s user 0.009s sys 0.000s
I20260924 00:20:57.806944 11039 raft_consensus.cc:348] T b87bc7d0b65e4153aa649444fcc2c128 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:57.807654 11039 raft_consensus.cc:374] T b87bc7d0b65e4153aa649444fcc2c128 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:57.807914 11039 raft_consensus.cc:729] T b87bc7d0b65e4153aa649444fcc2c128 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:20:57.808494 11039 consensus_queue.cc:260] T b87bc7d0b65e4153aa649444fcc2c128 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:57.808992 10371 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_092022b22e2f4c378fa31f8d19f05029"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:20:57.810334 11039 ts_tablet_manager.cc:1428] T b87bc7d0b65e4153aa649444fcc2c128 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260924 00:20:57.810631 10371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_092022b22e2f4c378fa31f8d19f05029 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:57.815287 10778 tablet_service.cc:1467] Processing CreateTablet for tablet 86374d72238d42c29542e649a7a0200d (DEFAULT_TABLE table=default.Table_6a3c73b6e78a4066bb59da236b32ab81 [id=8a99c64662ce4d3bba44aeb578c55c16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:57.816395 10778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86374d72238d42c29542e649a7a0200d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:57.817435 10917 tablet_service.cc:1467] Processing CreateTablet for tablet 96960f3061fb43e9939b470f49edf1e9 (DEFAULT_TABLE table=default.Table_9abf361701cd4e6faee21d3e41dfeae4 [id=6251651eb5944ee086dd564d86204148]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:57.818512 10917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96960f3061fb43e9939b470f49edf1e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:57.820052 10776 tablet_service.cc:1467] Processing CreateTablet for tablet 85a0ad61b7cb4e11b30e84ddc4655b64 (DEFAULT_TABLE table=default.Table_f1d482834246412aab6e26a93c7fe32c [id=12285914811f4220822b47a2a62b01ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:57.821781 10776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85a0ad61b7cb4e11b30e84ddc4655b64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:57.822635 10366 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_f1d482834246412aab6e26a93c7fe32c" } new_table_name: "default.Table_b8d64d06746d4e1f8f357136d1cebfe9" modify_external_catalogs: true
I20260924 00:20:57.824044 10921 tablet_service.cc:1467] Processing CreateTablet for tablet 86374d72238d42c29542e649a7a0200d (DEFAULT_TABLE table=default.Table_6a3c73b6e78a4066bb59da236b32ab81 [id=8a99c64662ce4d3bba44aeb578c55c16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:57.825608 10921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86374d72238d42c29542e649a7a0200d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:57.832252 10919 tablet_service.cc:1514] Processing DeleteTablet for tablet b87bc7d0b65e4153aa649444fcc2c128 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:56 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:20:57.840317 10919 tablet_service.cc:1467] Processing CreateTablet for tablet 85a0ad61b7cb4e11b30e84ddc4655b64 (DEFAULT_TABLE table=default.Table_f1d482834246412aab6e26a93c7fe32c [id=12285914811f4220822b47a2a62b01ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:57.841573 10919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85a0ad61b7cb4e11b30e84ddc4655b64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:57.846889 11040 tablet_bootstrap.cc:492] T 5f84857aad964b01b94e69af13007a55 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:20:57.847389 11040 ts_tablet_manager.cc:1397] T 5f84857aad964b01b94e69af13007a55 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.058s user 0.009s sys 0.000s
I20260924 00:20:57.849489 11040 raft_consensus.cc:348] T 5f84857aad964b01b94e69af13007a55 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:57.849713 10365 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_9c2aa717e62340c785be379c0dd5837c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:20:57.851797 10365 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9c2aa717e62340c785be379c0dd5837c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:57.853590 11040 raft_consensus.cc:374] T 5f84857aad964b01b94e69af13007a55 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:57.853850 11040 raft_consensus.cc:729] T 5f84857aad964b01b94e69af13007a55 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:20:57.854559 11040 consensus_queue.cc:260] T 5f84857aad964b01b94e69af13007a55 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:57.861677 11051 ts_tablet_manager.cc:1918] T 720e1a36bcfb4260b3689d99a7358205 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:57.868367 11040 ts_tablet_manager.cc:1428] T 5f84857aad964b01b94e69af13007a55 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.021s user 0.005s sys 0.000s
I20260924 00:20:57.873760 11040 tablet_bootstrap.cc:492] T 9a288e9b6f804283b6d48e4d4e1dd828 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:20:57.881186 11051 log.cc:1196] T 720e1a36bcfb4260b3689d99a7358205 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/720e1a36bcfb4260b3689d99a7358205
I20260924 00:20:57.882000 11051 ts_tablet_manager.cc:1939] T 720e1a36bcfb4260b3689d99a7358205 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:20:57.882862 11040 tablet_bootstrap.cc:654] T 9a288e9b6f804283b6d48e4d4e1dd828 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:57.893219 11040 tablet_bootstrap.cc:492] T 9a288e9b6f804283b6d48e4d4e1dd828 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
W20260924 00:20:57.893290 10359 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.126:34479, user_credentials={real_user=slave}} blocked reactor thread for 131356us
I20260924 00:20:57.893680 11040 ts_tablet_manager.cc:1397] T 9a288e9b6f804283b6d48e4d4e1dd828 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.020s user 0.009s sys 0.005s
I20260924 00:20:57.895663 11040 raft_consensus.cc:348] T 9a288e9b6f804283b6d48e4d4e1dd828 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:57.896271 11040 raft_consensus.cc:374] T 9a288e9b6f804283b6d48e4d4e1dd828 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:57.896512 11040 raft_consensus.cc:729] T 9a288e9b6f804283b6d48e4d4e1dd828 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:20:57.897574 10633 tablet_service.cc:1467] Processing CreateTablet for tablet 5d7d52be4f1f4fc29b8172b9fa878f76 (DEFAULT_TABLE table=default.Table_451536c1b7444188b1dcab7ce70d663f [id=fde3781ccd3748f2acc7ac5f3f5064bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:57.898236 10634 tablet_service.cc:1467] Processing CreateTablet for tablet 86374d72238d42c29542e649a7a0200d (DEFAULT_TABLE table=default.Table_6a3c73b6e78a4066bb59da236b32ab81 [id=8a99c64662ce4d3bba44aeb578c55c16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:57.898610 10633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d7d52be4f1f4fc29b8172b9fa878f76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:57.898392 10632 tablet_service.cc:1467] Processing CreateTablet for tablet d44932f8f08e4ea7acc05b8ec511f045 (DEFAULT_TABLE table=default.Table_321e6959ada84ccea6413114309bc3b8 [id=8092ca5df917480aba7e26a4cedf9fbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:57.899605 10635 tablet_service.cc:1467] Processing CreateTablet for tablet f294831873d548e4b9201c1bbb964096 (DEFAULT_TABLE table=default.Table_bc2eec8db7494414acf50558e06cd11e [id=e2cdad6dfe7846ebb3c14f5611cf0f18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:57.902124 10631 tablet_service.cc:1467] Processing CreateTablet for tablet 85a0ad61b7cb4e11b30e84ddc4655b64 (DEFAULT_TABLE table=default.Table_f1d482834246412aab6e26a93c7fe32c [id=12285914811f4220822b47a2a62b01ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:57.900722 10636 tablet_service.cc:1467] Processing CreateTablet for tablet 96960f3061fb43e9939b470f49edf1e9 (DEFAULT_TABLE table=default.Table_9abf361701cd4e6faee21d3e41dfeae4 [id=6251651eb5944ee086dd564d86204148]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:57.907779 10367 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 453ce17089ce44ba8da67a0893301c3c
I20260924 00:20:57.909147 10367 catalog_manager.cc:6565] ReplaceTablet: tablet 453ce17089ce44ba8da67a0893301c3c deleted and replaced by tablet 195b033091b543b58ee08417eae2c0e6
I20260924 00:20:57.908846 11040 consensus_queue.cc:260] T 9a288e9b6f804283b6d48e4d4e1dd828 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:57.911348 10778 tablet_service.cc:1514] Processing DeleteTablet for tablet 453ce17089ce44ba8da67a0893301c3c with delete_type TABLET_DATA_DELETED (replaced by tablet 195b033091b543b58ee08417eae2c0e6) from {username='slave'} at 127.0.0.1:35996
I20260924 00:20:57.911926 10634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86374d72238d42c29542e649a7a0200d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:57.919819 10630 tablet_service.cc:1514] Processing DeleteTablet for tablet 453ce17089ce44ba8da67a0893301c3c with delete_type TABLET_DATA_DELETED (replaced by tablet 195b033091b543b58ee08417eae2c0e6) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:57.921734 10635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f294831873d548e4b9201c1bbb964096. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:57.925577 10636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96960f3061fb43e9939b470f49edf1e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:57.929615 10632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d44932f8f08e4ea7acc05b8ec511f045. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:57.930063 10367 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_e83ceabd11404e14bb1eee103fffaaed"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:20:57.931890 10367 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e83ceabd11404e14bb1eee103fffaaed in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:57.933720 11040 ts_tablet_manager.cc:1428] T 9a288e9b6f804283b6d48e4d4e1dd828 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.040s user 0.005s sys 0.000s
I20260924 00:20:57.934592 10631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85a0ad61b7cb4e11b30e84ddc4655b64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:57.940522 10372 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet c212525586544d8f9d92bc4b8bd8acaa
I20260924 00:20:57.944118 10778 tablet_service.cc:1514] Processing DeleteTablet for tablet c212525586544d8f9d92bc4b8bd8acaa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:57 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:20:57.948236 10630 tablet_service.cc:1514] Processing DeleteTablet for tablet c212525586544d8f9d92bc4b8bd8acaa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:57 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:57.949211 10373 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 5d7d52be4f1f4fc29b8172b9fa878f76
I20260924 00:20:57.950671 10778 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d7d52be4f1f4fc29b8172b9fa878f76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:57 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:20:57.981206 10635 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d7d52be4f1f4fc29b8172b9fa878f76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:57 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:57.987365 10373 master_service.cc:946] ReplaceTablet: received request to replace tablet bcd8a6b92b8e492d96994195fa7f3ae5 from {username='slave'} at 127.0.0.1:47022
I20260924 00:20:57.996515 11040 tablet_bootstrap.cc:492] T 732a45a824274bdda8f7c060723a8cd2 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:20:58.010288 11041 tablet_bootstrap.cc:492] T b87bc7d0b65e4153aa649444fcc2c128 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:20:58.010695 11041 ts_tablet_manager.cc:1397] T b87bc7d0b65e4153aa649444fcc2c128 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.453s user 0.020s sys 0.009s
I20260924 00:20:58.013543 11040 tablet_bootstrap.cc:654] T 732a45a824274bdda8f7c060723a8cd2 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
W20260924 00:20:58.016475 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet af2f58e6757a4b738a8ec38b2e228f9f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet af2f58e6757a4b738a8ec38b2e228f9f already in progress: creating tablet
I20260924 00:20:58.017336 11050 tablet_replica.cc:330] T 1e5beeac2a1d4cb18b5d3d2408fd574b P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:20:58.017978 11050 raft_consensus.cc:2229] T 1e5beeac2a1d4cb18b5d3d2408fd574b P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:20:58.019104 11050 raft_consensus.cc:2258] T 1e5beeac2a1d4cb18b5d3d2408fd574b P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:58.029070 11041 raft_consensus.cc:348] T b87bc7d0b65e4153aa649444fcc2c128 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:58.029711 11041 raft_consensus.cc:374] T b87bc7d0b65e4153aa649444fcc2c128 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:58.029974 11041 raft_consensus.cc:729] T b87bc7d0b65e4153aa649444fcc2c128 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:20:57.880668 11079 raft_consensus.cc:482] T a1a79421de4049aea258261ff9bd00b4 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:20:58.033281 11079 raft_consensus.cc:504] T a1a79421de4049aea258261ff9bd00b4 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:58.037305 11050 ts_tablet_manager.cc:1905] T 1e5beeac2a1d4cb18b5d3d2408fd574b P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:58.070047 10922 tablet_service.cc:1514] Processing DeleteTablet for tablet c212525586544d8f9d92bc4b8bd8acaa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:57 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:20:58.070582 10915 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d7d52be4f1f4fc29b8172b9fa878f76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:57 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:20:58.095163 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 720e1a36bcfb4260b3689d99a7358205 (table default.Table_b1ae3ebb4b3e4c43a1a684b765cfdd38 [id=9d8bc2174e884f97b364cc1d2e5af3b4]) successfully deleted
I20260924 00:20:58.097348 11039 tablet_bootstrap.cc:492] T c83942a74816465fa7f3f2c3e18c98bb P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:20:58.105001 11051 tablet_replica.cc:330] T a1a79421de4049aea258261ff9bd00b4 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:20:58.104997 11050 ts_tablet_manager.cc:1918] T 1e5beeac2a1d4cb18b5d3d2408fd574b P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20260924 00:20:58.105353 11050 log.cc:1196] T 1e5beeac2a1d4cb18b5d3d2408fd574b P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/1e5beeac2a1d4cb18b5d3d2408fd574b
I20260924 00:20:58.106032 11050 ts_tablet_manager.cc:1939] T 1e5beeac2a1d4cb18b5d3d2408fd574b P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:20:58.116891 11051 raft_consensus.cc:2229] T a1a79421de4049aea258261ff9bd00b4 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:58.158413 11051 raft_consensus.cc:2258] T a1a79421de4049aea258261ff9bd00b4 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:58.157049 11079 leader_election.cc:288] T a1a79421de4049aea258261ff9bd00b4 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:20:58.160260 11079 raft_consensus.cc:482] T 5f84857aad964b01b94e69af13007a55 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:20:58.160737 11079 raft_consensus.cc:504] T 5f84857aad964b01b94e69af13007a55 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:58.162930 11079 leader_election.cc:288] T 5f84857aad964b01b94e69af13007a55 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:20:58.174130 11049 ts_tablet_manager.cc:1918] T 9a288e9b6f804283b6d48e4d4e1dd828 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:58.174611 11049 log.cc:1196] T 9a288e9b6f804283b6d48e4d4e1dd828 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/9a288e9b6f804283b6d48e4d4e1dd828
I20260924 00:20:58.175290 11049 ts_tablet_manager.cc:1939] T 9a288e9b6f804283b6d48e4d4e1dd828 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:20:58.176669 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 9a288e9b6f804283b6d48e4d4e1dd828 (table default.Table_5f8c295237ab4e0ea184a03a9fa458a2 [id=be698d8b82734ff5940f6915ba1d4df1]) successfully deleted
W20260924 00:20:58.178355 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet af2f58e6757a4b738a8ec38b2e228f9f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet af2f58e6757a4b738a8ec38b2e228f9f already in progress: creating tablet
I20260924 00:20:58.182588 11051 ts_tablet_manager.cc:1905] T a1a79421de4049aea258261ff9bd00b4 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:58.196973 10372 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_6a3c73b6e78a4066bb59da236b32ab81" } modify_external_catalogs: true
I20260924 00:20:58.197576 10372 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_6a3c73b6e78a4066bb59da236b32ab81" } modify_external_catalogs: true
I20260924 00:20:58.201902 10374 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_99da18207fc542fbb68972724c4d410e" } modify_external_catalogs: true
I20260924 00:20:58.202503 10374 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_99da18207fc542fbb68972724c4d410e" } modify_external_catalogs: true
I20260924 00:20:58.213688 10916 tablet_service.cc:1514] Processing DeleteTablet for tablet 453ce17089ce44ba8da67a0893301c3c with delete_type TABLET_DATA_DELETED (replaced by tablet 195b033091b543b58ee08417eae2c0e6) from {username='slave'} at 127.0.0.1:35358
I20260924 00:20:58.218652 11039 tablet_bootstrap.cc:654] T c83942a74816465fa7f3f2c3e18c98bb P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:58.227435 11041 consensus_queue.cc:260] T b87bc7d0b65e4153aa649444fcc2c128 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:58.245182 11144 raft_consensus.cc:482] T b87bc7d0b65e4153aa649444fcc2c128 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:20:58.245806 11144 raft_consensus.cc:504] T b87bc7d0b65e4153aa649444fcc2c128 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:58.248289 11144 leader_election.cc:288] T b87bc7d0b65e4153aa649444fcc2c128 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:20:58.248998 11049 tablet_replica.cc:330] T 1e5beeac2a1d4cb18b5d3d2408fd574b P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:20:58.249742 11049 raft_consensus.cc:2229] T 1e5beeac2a1d4cb18b5d3d2408fd574b P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:58.250478 11049 raft_consensus.cc:2258] T 1e5beeac2a1d4cb18b5d3d2408fd574b P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:58.252501 11049 ts_tablet_manager.cc:1905] T 1e5beeac2a1d4cb18b5d3d2408fd574b P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260924 00:20:58.292410 10370 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ccb26e0fbc1948bc9a00e9e26c4d1139 consensus state has no leader
I20260924 00:20:58.293305 10370 master_service.cc:946] ReplaceTablet: received request to replace tablet c212525586544d8f9d92bc4b8bd8acaa from {username='slave'} at 127.0.0.1:47022
W20260924 00:20:58.294440 10366 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 85a0ad61b7cb4e11b30e84ddc4655b64 consensus state has no leader
I20260924 00:20:58.296204 10366 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_a6631f3ce9a04312b19aa2e85b4c9fe3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:20:58.301893 10366 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a6631f3ce9a04312b19aa2e85b4c9fe3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:58.324899 11039 tablet_bootstrap.cc:492] T c83942a74816465fa7f3f2c3e18c98bb P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:20:58.283591 11041 ts_tablet_manager.cc:1428] T b87bc7d0b65e4153aa649444fcc2c128 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.271s user 0.005s sys 0.002s
I20260924 00:20:58.326987 11041 tablet_bootstrap.cc:492] T c83942a74816465fa7f3f2c3e18c98bb P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:20:58.332185 11041 tablet_bootstrap.cc:654] T c83942a74816465fa7f3f2c3e18c98bb P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
W20260924 00:20:58.332536 10361 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.124:35345, user_credentials={real_user=slave}} blocked reactor thread for 91574.4us
I20260924 00:20:58.333361 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 1e5beeac2a1d4cb18b5d3d2408fd574b (table default.Table_a5d798f7819e403a96f27e98cb0af2bb [id=30a3f7c8fb9c46cf8b883eedae25d938]) successfully deleted
W20260924 00:20:58.334157 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 453ce17089ce44ba8da67a0893301c3c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 453ce17089ce44ba8da67a0893301c3c already in progress: creating tablet
W20260924 00:20:58.334998 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet c212525586544d8f9d92bc4b8bd8acaa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c212525586544d8f9d92bc4b8bd8acaa already in progress: creating tablet
W20260924 00:20:58.335665 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 5d7d52be4f1f4fc29b8172b9fa878f76 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5d7d52be4f1f4fc29b8172b9fa878f76 already in progress: creating tablet
I20260924 00:20:58.337337 11039 ts_tablet_manager.cc:1397] T c83942a74816465fa7f3f2c3e18c98bb P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.240s user 0.007s sys 0.003s
W20260924 00:20:58.356472 10876 leader_election.cc:341] T a1a79421de4049aea258261ff9bd00b4 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: a1a79421de4049aea258261ff9bd00b4
W20260924 00:20:58.384516 10361 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 85a0ad61b7cb4e11b30e84ddc4655b64 (table default.Table_b8d64d06746d4e1f8f357136d1cebfe9 [id=12285914811f4220822b47a2a62b01ab], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 85a0ad61b7cb4e11b30e84ddc4655b64 consensus state has no leader
I20260924 00:20:58.340471 11039 raft_consensus.cc:348] T c83942a74816465fa7f3f2c3e18c98bb P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:58.395166 11039 raft_consensus.cc:374] T c83942a74816465fa7f3f2c3e18c98bb P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:58.395627 11039 raft_consensus.cc:729] T c83942a74816465fa7f3f2c3e18c98bb P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:20:58.396392 11039 consensus_queue.cc:260] T c83942a74816465fa7f3f2c3e18c98bb P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:58.322538 11146 raft_consensus.cc:482] T 5f84857aad964b01b94e69af13007a55 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:20:58.354599 10655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f84857aad964b01b94e69af13007a55" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:20:58.355362 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b87bc7d0b65e4153aa649444fcc2c128" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:20:58.411688 11068 raft_consensus.cc:482] T ad256de63e0948928a51d9231152ac2d P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:20:58.413815 10654 raft_consensus.cc:2454] T b87bc7d0b65e4153aa649444fcc2c128 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
I20260924 00:20:58.413515 11146 raft_consensus.cc:504] T 5f84857aad964b01b94e69af13007a55 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:58.414258 11068 raft_consensus.cc:504] T ad256de63e0948928a51d9231152ac2d P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:58.414916 10655 raft_consensus.cc:2454] T 5f84857aad964b01b94e69af13007a55 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
I20260924 00:20:58.415486 11146 leader_election.cc:288] T 5f84857aad964b01b94e69af13007a55 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:20:58.416011 11068 leader_election.cc:288] T ad256de63e0948928a51d9231152ac2d P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:20:58.417511 10876 leader_election.cc:302] T b87bc7d0b65e4153aa649444fcc2c128 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:20:58.418412 10876 leader_election.cc:302] T 5f84857aad964b01b94e69af13007a55 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:20:58.419024 11040 tablet_bootstrap.cc:492] T 732a45a824274bdda8f7c060723a8cd2 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:20:58.419394 11040 ts_tablet_manager.cc:1397] T 732a45a824274bdda8f7c060723a8cd2 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.449s user 0.094s sys 0.117s
I20260924 00:20:58.419622 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad256de63e0948928a51d9231152ac2d" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:20:58.420360 10942 raft_consensus.cc:2379] T ad256de63e0948928a51d9231152ac2d P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48fbd603abe146c585d15196e1b5be08 in current term 1: Already voted for candidate b10a9082a37b46f09d581944914c1670 in this term.
I20260924 00:20:58.419643 10941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f84857aad964b01b94e69af13007a55" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:20:58.421633 10941 raft_consensus.cc:2454] T 5f84857aad964b01b94e69af13007a55 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b10a9082a37b46f09d581944914c1670 in term 0.
I20260924 00:20:58.421702 11144 raft_consensus.cc:2781] T b87bc7d0b65e4153aa649444fcc2c128 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:20:58.422022 11144 raft_consensus.cc:482] T b87bc7d0b65e4153aa649444fcc2c128 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:20:58.422283 11144 raft_consensus.cc:3037] T b87bc7d0b65e4153aa649444fcc2c128 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:20:58.422704 10589 leader_election.cc:302] T 5f84857aad964b01b94e69af13007a55 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:20:58.423370 11146 raft_consensus.cc:2781] T 5f84857aad964b01b94e69af13007a55 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:20:58.423658 11146 raft_consensus.cc:482] T 5f84857aad964b01b94e69af13007a55 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:20:58.423696 11079 raft_consensus.cc:2781] T 5f84857aad964b01b94e69af13007a55 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:20:58.423977 11146 raft_consensus.cc:3037] T 5f84857aad964b01b94e69af13007a55 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:20:58.423993 11079 raft_consensus.cc:482] T 5f84857aad964b01b94e69af13007a55 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:20:58.424259 11079 raft_consensus.cc:3037] T 5f84857aad964b01b94e69af13007a55 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:20:58.429673 11144 raft_consensus.cc:504] T b87bc7d0b65e4153aa649444fcc2c128 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:58.429963 10655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad256de63e0948928a51d9231152ac2d" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:20:58.432145 10733 leader_election.cc:302] T ad256de63e0948928a51d9231152ac2d P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08; no voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef
I20260924 00:20:58.433234 11040 raft_consensus.cc:348] T 732a45a824274bdda8f7c060723a8cd2 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:58.433693 10655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b87bc7d0b65e4153aa649444fcc2c128" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:20:58.434010 11040 raft_consensus.cc:374] T 732a45a824274bdda8f7c060723a8cd2 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:58.434175 10655 raft_consensus.cc:3037] T b87bc7d0b65e4153aa649444fcc2c128 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:20:58.434248 11040 raft_consensus.cc:729] T 732a45a824274bdda8f7c060723a8cd2 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:20:58.434346 10800 raft_consensus.cc:3037] T ad256de63e0948928a51d9231152ac2d P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:20:58.434818 11040 consensus_queue.cc:260] T 732a45a824274bdda8f7c060723a8cd2 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:58.437503 11039 ts_tablet_manager.cc:1428] T c83942a74816465fa7f3f2c3e18c98bb P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.100s user 0.005s sys 0.001s
I20260924 00:20:58.438324 11039 tablet_bootstrap.cc:492] T 732a45a824274bdda8f7c060723a8cd2 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:20:58.441629 10655 raft_consensus.cc:2454] T b87bc7d0b65e4153aa649444fcc2c128 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 1.
I20260924 00:20:58.442775 10876 leader_election.cc:302] T b87bc7d0b65e4153aa649444fcc2c128 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:20:58.442603 11146 raft_consensus.cc:504] T 5f84857aad964b01b94e69af13007a55 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:58.444424 11146 leader_election.cc:288] T 5f84857aad964b01b94e69af13007a55 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Requested vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:20:58.446266 10941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f84857aad964b01b94e69af13007a55" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef"
I20260924 00:20:58.360844 11051 ts_tablet_manager.cc:1918] T a1a79421de4049aea258261ff9bd00b4 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:58.454165 11051 log.cc:1196] T a1a79421de4049aea258261ff9bd00b4 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/a1a79421de4049aea258261ff9bd00b4
I20260924 00:20:58.454383 11041 tablet_bootstrap.cc:492] T c83942a74816465fa7f3f2c3e18c98bb P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:20:58.455176 11144 leader_election.cc:288] T b87bc7d0b65e4153aa649444fcc2c128 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:20:58.456017 10799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f84857aad964b01b94e69af13007a55" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:20:58.456617 10799 raft_consensus.cc:2454] T 5f84857aad964b01b94e69af13007a55 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b10a9082a37b46f09d581944914c1670 in term 0.
I20260924 00:20:58.456774 11144 raft_consensus.cc:2781] T b87bc7d0b65e4153aa649444fcc2c128 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:20:58.458355 11144 raft_consensus.cc:686] T b87bc7d0b65e4153aa649444fcc2c128 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Becoming Leader. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Running, Role: LEADER
I20260924 00:20:58.459154 11144 consensus_queue.cc:237] T b87bc7d0b65e4153aa649444fcc2c128 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:58.459540 10799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f84857aad964b01b94e69af13007a55" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:20:58.465086 11041 ts_tablet_manager.cc:1397] T c83942a74816465fa7f3f2c3e18c98bb P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.138s user 0.007s sys 0.003s
I20260924 00:20:58.465104 11051 ts_tablet_manager.cc:1939] T a1a79421de4049aea258261ff9bd00b4 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:20:58.466478 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet a1a79421de4049aea258261ff9bd00b4 (table default.Table_b1726c09065e4fbaa2e6799b3ebdbc74 [id=4dbb68e0a32349ad8da2945db650584d]) successfully deleted
I20260924 00:20:58.467340 11079 raft_consensus.cc:504] T 5f84857aad964b01b94e69af13007a55 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:58.468477 10941 raft_consensus.cc:2379] T 5f84857aad964b01b94e69af13007a55 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b10a9082a37b46f09d581944914c1670 in current term 1: Already voted for candidate c8626e300d0644b0908797ad61bd7cef in this term.
I20260924 00:20:58.469059 11051 tablet_replica.cc:330] T 5f84857aad964b01b94e69af13007a55 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:20:58.471724 11049 ts_tablet_manager.cc:1918] T 1e5beeac2a1d4cb18b5d3d2408fd574b P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20260924 00:20:58.473677 11051 raft_consensus.cc:2229] T 5f84857aad964b01b94e69af13007a55 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:58.474134 11051 raft_consensus.cc:2258] T 5f84857aad964b01b94e69af13007a55 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:58.472103 11079 leader_election.cc:288] T 5f84857aad964b01b94e69af13007a55 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:20:58.481618 10367 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet c83942a74816465fa7f3f2c3e18c98bb (table default.Table_4010035427fa43afae7f802f2dadf83c [id=7eb84ebbe346450eb2b944db685c414b]). Expected version 2 got 0
I20260924 00:20:58.491786 10921 tablet_service.cc:1467] Processing CreateTablet for tablet 59c077eb68794340ac900dbedc7056ee (DEFAULT_TABLE table=default.Table_5cf772cb831f4e7e97430617b962db80 [id=226555984f9741d3955c34512885c9cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:58.492872 10921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59c077eb68794340ac900dbedc7056ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:58.475765 11039 tablet_bootstrap.cc:654] T 732a45a824274bdda8f7c060723a8cd2 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:58.496487 10916 tablet_service.cc:1467] Processing CreateTablet for tablet 7463fcb18e884b1dbc478ebaa19f6cb4 (DEFAULT_TABLE table=default.Table_f21f079a72ef4c6ca5376a661c2906e8 [id=8e2fe68fe5da4414b7269f526b8af683]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:58.497488 10916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7463fcb18e884b1dbc478ebaa19f6cb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:58.467347 11041 raft_consensus.cc:348] T c83942a74816465fa7f3f2c3e18c98bb P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:58.497828 11049 log.cc:1196] T 1e5beeac2a1d4cb18b5d3d2408fd574b P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/1e5beeac2a1d4cb18b5d3d2408fd574b
I20260924 00:20:58.497845 11041 raft_consensus.cc:374] T c83942a74816465fa7f3f2c3e18c98bb P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:58.498198 11041 raft_consensus.cc:729] T c83942a74816465fa7f3f2c3e18c98bb P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:20:58.498553 11049 ts_tablet_manager.cc:1939] T 1e5beeac2a1d4cb18b5d3d2408fd574b P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:20:58.498760 11041 consensus_queue.cc:260] T c83942a74816465fa7f3f2c3e18c98bb P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:58.500577 11041 ts_tablet_manager.cc:1428] T c83942a74816465fa7f3f2c3e18c98bb P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.035s user 0.004s sys 0.001s
I20260924 00:20:58.500361 10920 tablet_service.cc:1467] Processing CreateTablet for tablet 666755ec87bb496db75e43ec9204ec5b (DEFAULT_TABLE table=default.Table_fcb196dda8a044e8b443fa810c5b8d41 [id=00b8c1d674cc460bb7b2425c55037246]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:58.501309 10920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 666755ec87bb496db75e43ec9204ec5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:58.502382 11041 tablet_bootstrap.cc:492] T 732a45a824274bdda8f7c060723a8cd2 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:20:58.505657 10918 tablet_service.cc:1467] Processing CreateTablet for tablet fcf651adb1d3459facfe3a8df2a91ed7 (DEFAULT_TABLE table=default.Table_c879109e72984beb85e10878ed02c18f [id=0912091b30374df895fec45a7eafe4cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:58.506659 10918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcf651adb1d3459facfe3a8df2a91ed7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:58.507115 11041 tablet_bootstrap.cc:654] T 732a45a824274bdda8f7c060723a8cd2 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:58.509703 10917 tablet_service.cc:1467] Processing CreateTablet for tablet 5b0345252c3c496db2987c30adce3da7 (DEFAULT_TABLE table=default.Table_a5d798f7819e403a96f27e98cb0af2bb [id=30a3f7c8fb9c46cf8b883eedae25d938]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:58.511059 10917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b0345252c3c496db2987c30adce3da7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:58.516747 10370 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_f21f079a72ef4c6ca5376a661c2906e8" } new_table_name: "default.Table_610074e75ae343618ae2141e342529b9" modify_external_catalogs: true
I20260924 00:20:58.521164 10371 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_89e5d14641754be1934c7e3c880b21ee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:20:58.520757 10365 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_4010035427fa43afae7f802f2dadf83c" } new_table_name: "default.Table_41d1376556a9466fa348723fd8e2b2be" modify_external_catalogs: true
W20260924 00:20:58.522955 10371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_89e5d14641754be1934c7e3c880b21ee in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:58.532058 11041 tablet_bootstrap.cc:492] T 732a45a824274bdda8f7c060723a8cd2 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:20:58.532754 11041 ts_tablet_manager.cc:1397] T 732a45a824274bdda8f7c060723a8cd2 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.030s user 0.006s sys 0.004s
I20260924 00:20:58.529210 11049 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 52) took 1012 ms. Trace:
I20260924 00:20:58.536659 11049 rpcz_store.cc:276] 0924 00:20:57.516638 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:20:57.516750 (+ 112us) service_pool.cc:227] Handling call
0924 00:20:58.248763 (+732013us) ts_tablet_manager.cc:1145] Deleting tablet 1e5beeac2a1d4cb18b5d3d2408fd574b
0924 00:20:58.248837 (+ 74us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:20:58.445287 (+196450us) tablet_metadata.cc:641] Metadata flushed
0924 00:20:58.471509 (+ 26222us) tablet_metadata.cc:641] Metadata flushed
0924 00:20:58.528972 (+ 57463us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":23066,"tablet-delete.queue_time_us":661894}
I20260924 00:20:58.541623 11041 raft_consensus.cc:348] T 732a45a824274bdda8f7c060723a8cd2 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:58.547957 11041 raft_consensus.cc:374] T 732a45a824274bdda8f7c060723a8cd2 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:58.550066 11041 raft_consensus.cc:729] T 732a45a824274bdda8f7c060723a8cd2 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:20:58.558108 11041 consensus_queue.cc:260] T 732a45a824274bdda8f7c060723a8cd2 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:58.566210 10368 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet c83942a74816465fa7f3f2c3e18c98bb (table default.Table_4010035427fa43afae7f802f2dadf83c [id=7eb84ebbe346450eb2b944db685c414b]). Expected version 2 got 0
I20260924 00:20:58.567812 11041 ts_tablet_manager.cc:1428] T 732a45a824274bdda8f7c060723a8cd2 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.035s user 0.001s sys 0.005s
I20260924 00:20:58.577049 11041 tablet_bootstrap.cc:492] T 1877e10d1db845f2a001063fb3bffef4 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:20:58.578166 10367 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_593b4d43f8144c04ab048e8221b3fc8e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:20:58.581115 10367 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_593b4d43f8144c04ab048e8221b3fc8e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:58.586055 11041 tablet_bootstrap.cc:654] T 1877e10d1db845f2a001063fb3bffef4 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:58.586136 10366 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_3f0fdced96874ff3abd25f14bb25fa86"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:20:58.588390 10366 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3f0fdced96874ff3abd25f14bb25fa86 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:58.595184 10634 tablet_service.cc:1467] Processing CreateTablet for tablet 59c077eb68794340ac900dbedc7056ee (DEFAULT_TABLE table=default.Table_5cf772cb831f4e7e97430617b962db80 [id=226555984f9741d3955c34512885c9cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:58.595809 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 1e5beeac2a1d4cb18b5d3d2408fd574b (table default.Table_a5d798f7819e403a96f27e98cb0af2bb [id=30a3f7c8fb9c46cf8b883eedae25d938]) successfully deleted
I20260924 00:20:58.596215 10634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59c077eb68794340ac900dbedc7056ee. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:20:58.596553 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 453ce17089ce44ba8da67a0893301c3c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 453ce17089ce44ba8da67a0893301c3c already in progress: creating tablet
W20260924 00:20:58.597229 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet c212525586544d8f9d92bc4b8bd8acaa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c212525586544d8f9d92bc4b8bd8acaa already in progress: creating tablet
W20260924 00:20:58.597811 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 5d7d52be4f1f4fc29b8172b9fa878f76 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5d7d52be4f1f4fc29b8172b9fa878f76 already in progress: creating tablet
I20260924 00:20:58.597636 10636 tablet_service.cc:1467] Processing CreateTablet for tablet 7463fcb18e884b1dbc478ebaa19f6cb4 (DEFAULT_TABLE table=default.Table_f21f079a72ef4c6ca5376a661c2906e8 [id=8e2fe68fe5da4414b7269f526b8af683]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:58.598601 10636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7463fcb18e884b1dbc478ebaa19f6cb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:58.601243 10369 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet c83942a74816465fa7f3f2c3e18c98bb (table default.Table_4010035427fa43afae7f802f2dadf83c [id=7eb84ebbe346450eb2b944db685c414b]). Expected version 2 got 0
I20260924 00:20:58.601653 10631 tablet_service.cc:1467] Processing CreateTablet for tablet 666755ec87bb496db75e43ec9204ec5b (DEFAULT_TABLE table=default.Table_fcb196dda8a044e8b443fa810c5b8d41 [id=00b8c1d674cc460bb7b2425c55037246]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:58.602658 10631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 666755ec87bb496db75e43ec9204ec5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:58.605664 10632 tablet_service.cc:1467] Processing CreateTablet for tablet fcf651adb1d3459facfe3a8df2a91ed7 (DEFAULT_TABLE table=default.Table_c879109e72984beb85e10878ed02c18f [id=0912091b30374df895fec45a7eafe4cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:58.606671 10632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcf651adb1d3459facfe3a8df2a91ed7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:58.609236 11041 tablet_bootstrap.cc:492] T 1877e10d1db845f2a001063fb3bffef4 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:20:58.609620 11041 ts_tablet_manager.cc:1397] T 1877e10d1db845f2a001063fb3bffef4 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.041s user 0.005s sys 0.004s
I20260924 00:20:58.609575 10633 tablet_service.cc:1467] Processing CreateTablet for tablet 5b0345252c3c496db2987c30adce3da7 (DEFAULT_TABLE table=default.Table_a5d798f7819e403a96f27e98cb0af2bb [id=30a3f7c8fb9c46cf8b883eedae25d938]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:58.610836 10633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b0345252c3c496db2987c30adce3da7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:58.612032 11041 raft_consensus.cc:348] T 1877e10d1db845f2a001063fb3bffef4 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:58.612671 11041 raft_consensus.cc:374] T 1877e10d1db845f2a001063fb3bffef4 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:58.615686 10635 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f84857aad964b01b94e69af13007a55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:56 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:58.616382 11049 tablet_replica.cc:330] T 5f84857aad964b01b94e69af13007a55 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:20:58.617004 11049 raft_consensus.cc:2229] T 5f84857aad964b01b94e69af13007a55 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:58.617395 11049 raft_consensus.cc:2258] T 5f84857aad964b01b94e69af13007a55 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:58.618887 11049 ts_tablet_manager.cc:1905] T 5f84857aad964b01b94e69af13007a55 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:58.612890 11041 raft_consensus.cc:729] T 1877e10d1db845f2a001063fb3bffef4 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:20:58.627185 11041 consensus_queue.cc:260] T 1877e10d1db845f2a001063fb3bffef4 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:58.638405 11049 ts_tablet_manager.cc:1918] T 5f84857aad964b01b94e69af13007a55 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:58.497509 11051 ts_tablet_manager.cc:1905] T 5f84857aad964b01b94e69af13007a55 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:58.639696 11041 ts_tablet_manager.cc:1428] T 1877e10d1db845f2a001063fb3bffef4 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.030s user 0.006s sys 0.000s
I20260924 00:20:58.645829 11146 consensus_queue.cc:1035] T ad256de63e0948928a51d9231152ac2d P b10a9082a37b46f09d581944914c1670 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:20:58.657670 11041 tablet_bootstrap.cc:492] T 6ad838c884fc4b1d8f98781e7190e3df P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:20:58.460018 10799 raft_consensus.cc:3037] T 5f84857aad964b01b94e69af13007a55 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:20:58.662036 10372 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 86374d72238d42c29542e649a7a0200d
I20260924 00:20:58.663836 11049 log.cc:1196] T 5f84857aad964b01b94e69af13007a55 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/5f84857aad964b01b94e69af13007a55
I20260924 00:20:58.665542 11049 ts_tablet_manager.cc:1939] T 5f84857aad964b01b94e69af13007a55 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:20:58.669401 10774 tablet_service.cc:1467] Processing CreateTablet for tablet 7463fcb18e884b1dbc478ebaa19f6cb4 (DEFAULT_TABLE table=default.Table_f21f079a72ef4c6ca5376a661c2906e8 [id=8e2fe68fe5da4414b7269f526b8af683]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:58.670137 10778 tablet_service.cc:1467] Processing CreateTablet for tablet 666755ec87bb496db75e43ec9204ec5b (DEFAULT_TABLE table=default.Table_fcb196dda8a044e8b443fa810c5b8d41 [id=00b8c1d674cc460bb7b2425c55037246]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:58.670715 10776 tablet_service.cc:1467] Processing CreateTablet for tablet fcf651adb1d3459facfe3a8df2a91ed7 (DEFAULT_TABLE table=default.Table_c879109e72984beb85e10878ed02c18f [id=0912091b30374df895fec45a7eafe4cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:58.671377 10773 tablet_service.cc:1514] Processing DeleteTablet for tablet 86374d72238d42c29542e649a7a0200d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:58 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:20:58.671974 10780 tablet_service.cc:1467] Processing CreateTablet for tablet 5b0345252c3c496db2987c30adce3da7 (DEFAULT_TABLE table=default.Table_a5d798f7819e403a96f27e98cb0af2bb [id=30a3f7c8fb9c46cf8b883eedae25d938]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:58.672793 10777 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f84857aad964b01b94e69af13007a55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:56 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:20:58.673457 10779 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a288e9b6f804283b6d48e4d4e1dd828 with delete_type TABLET_DATA_DELETED (replaced by tablet 1877e10d1db845f2a001063fb3bffef4) from {username='slave'} at 127.0.0.1:35996
I20260924 00:20:58.681190 10635 tablet_service.cc:1514] Processing DeleteTablet for tablet 86374d72238d42c29542e649a7a0200d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:58 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:58.687420 10775 tablet_service.cc:1467] Processing CreateTablet for tablet 59c077eb68794340ac900dbedc7056ee (DEFAULT_TABLE table=default.Table_5cf772cb831f4e7e97430617b962db80 [id=226555984f9741d3955c34512885c9cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:58.684966 11068 raft_consensus.cc:2726] T ad256de63e0948928a51d9231152ac2d P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:20:58.688493 11041 tablet_bootstrap.cc:654] T 6ad838c884fc4b1d8f98781e7190e3df P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:58.696126 10775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59c077eb68794340ac900dbedc7056ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:58.698479 10799 raft_consensus.cc:2454] T 5f84857aad964b01b94e69af13007a55 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b10a9082a37b46f09d581944914c1670 in term 1.
W20260924 00:20:58.697062 10876 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:33461, user_credentials={real_user=slave}} blocked reactor thread for 254332us
I20260924 00:20:58.704162 10776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcf651adb1d3459facfe3a8df2a91ed7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:58.710904 10778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 666755ec87bb496db75e43ec9204ec5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:58.713534 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7463fcb18e884b1dbc478ebaa19f6cb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:58.717499 10780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b0345252c3c496db2987c30adce3da7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:58.735421 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b87bc7d0b65e4153aa649444fcc2c128" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:20:58.736989 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f84857aad964b01b94e69af13007a55" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:20:58.737375 10798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b87bc7d0b65e4153aa649444fcc2c128" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:20:58.737725 10797 raft_consensus.cc:2379] T 5f84857aad964b01b94e69af13007a55 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c8626e300d0644b0908797ad61bd7cef in current term 1: Already voted for candidate b10a9082a37b46f09d581944914c1670 in this term.
W20260924 00:20:58.739113 10878 leader_election.cc:341] T b87bc7d0b65e4153aa649444fcc2c128 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
W20260924 00:20:58.740160 10878 leader_election.cc:341] T b87bc7d0b65e4153aa649444fcc2c128 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:20:58.740127 10800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f84857aad964b01b94e69af13007a55" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:20:58.740887 10800 raft_consensus.cc:2379] T 5f84857aad964b01b94e69af13007a55 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c8626e300d0644b0908797ad61bd7cef in current term 1: Already voted for candidate b10a9082a37b46f09d581944914c1670 in this term.
W20260924 00:20:58.749583 10359 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.126:34479, user_credentials={real_user=slave}} blocked reactor thread for 66832us
W20260924 00:20:58.750211 10878 leader_election.cc:341] T a1a79421de4049aea258261ff9bd00b4 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Not found: Tablet not found: a1a79421de4049aea258261ff9bd00b4
I20260924 00:20:58.750440 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 5f84857aad964b01b94e69af13007a55 (table default.Table_d4b54eed2b8a4b3994b2635e01c5c273 [id=143547db8d2d4517877751965d3042ce]) successfully deleted
I20260924 00:20:58.750607 10878 leader_election.cc:302] T a1a79421de4049aea258261ff9bd00b4 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670
W20260924 00:20:58.751139 10878 raft_consensus.cc:2670] T a1a79421de4049aea258261ff9bd00b4 P c8626e300d0644b0908797ad61bd7cef: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260924 00:20:58.751148 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 86374d72238d42c29542e649a7a0200d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 86374d72238d42c29542e649a7a0200d already in progress: creating tablet
I20260924 00:20:58.753371 10373 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet bcd8a6b92b8e492d96994195fa7f3ae5
I20260924 00:20:58.754587 10373 catalog_manager.cc:6565] ReplaceTablet: tablet bcd8a6b92b8e492d96994195fa7f3ae5 deleted and replaced by tablet 0d555f059f84478eb805f4ca4534dab4
W20260924 00:20:58.775331 10370 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7463fcb18e884b1dbc478ebaa19f6cb4 consensus state has no leader
I20260924 00:20:58.777688 10916 tablet_service.cc:1514] Processing DeleteTablet for tablet bcd8a6b92b8e492d96994195fa7f3ae5 with delete_type TABLET_DATA_DELETED (replaced by tablet 0d555f059f84478eb805f4ca4534dab4) from {username='slave'} at 127.0.0.1:35358
W20260924 00:20:58.779301 10365 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c83942a74816465fa7f3f2c3e18c98bb consensus state has no leader
I20260924 00:20:58.788870 10369 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_6a2c714f26254ca89f7021633fc89fb7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:20:58.792884 10369 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6a2c714f26254ca89f7021633fc89fb7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:58.802543 10779 tablet_service.cc:1514] Processing DeleteTablet for tablet bcd8a6b92b8e492d96994195fa7f3ae5 with delete_type TABLET_DATA_DELETED (replaced by tablet 0d555f059f84478eb805f4ca4534dab4) from {username='slave'} at 127.0.0.1:35996
I20260924 00:20:58.802520 10773 tablet_service.cc:1514] Processing DeleteTablet for tablet 732a45a824274bdda8f7c060723a8cd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:56 UTC) from {username='slave'} at 127.0.0.1:35996
W20260924 00:20:58.806643 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 86374d72238d42c29542e649a7a0200d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 86374d72238d42c29542e649a7a0200d already in progress: creating tablet
I20260924 00:20:58.808674 11156 tablet_replica.cc:330] T 5f84857aad964b01b94e69af13007a55 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:20:58.828816 11039 tablet_bootstrap.cc:492] T 732a45a824274bdda8f7c060723a8cd2 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:20:58.829185 10918 tablet_service.cc:1514] Processing DeleteTablet for tablet 86374d72238d42c29542e649a7a0200d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:58 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:20:58.811836 11040 ts_tablet_manager.cc:1428] T 732a45a824274bdda8f7c060723a8cd2 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.392s user 0.015s sys 0.019s
I20260924 00:20:58.829337 11039 ts_tablet_manager.cc:1397] T 732a45a824274bdda8f7c060723a8cd2 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.391s user 0.003s sys 0.032s
I20260924 00:20:58.831538 11040 tablet_bootstrap.cc:492] T b87bc7d0b65e4153aa649444fcc2c128 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:20:58.834226 11039 raft_consensus.cc:348] T 732a45a824274bdda8f7c060723a8cd2 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:58.835075 11039 raft_consensus.cc:374] T 732a45a824274bdda8f7c060723a8cd2 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:58.844343 11156 raft_consensus.cc:2229] T 5f84857aad964b01b94e69af13007a55 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:58.844964 11156 raft_consensus.cc:2258] T 5f84857aad964b01b94e69af13007a55 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:58.857746 11039 raft_consensus.cc:729] T 732a45a824274bdda8f7c060723a8cd2 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:20:58.858502 10634 tablet_service.cc:1514] Processing DeleteTablet for tablet bcd8a6b92b8e492d96994195fa7f3ae5 with delete_type TABLET_DATA_DELETED (replaced by tablet 0d555f059f84478eb805f4ca4534dab4) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:58.860324 11146 consensus_queue.cc:1035] T ad256de63e0948928a51d9231152ac2d P b10a9082a37b46f09d581944914c1670 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260924 00:20:58.861780 10876 leader_election.cc:341] T 5f84857aad964b01b94e69af13007a55 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: 5f84857aad964b01b94e69af13007a55
I20260924 00:20:58.862222 10876 leader_election.cc:302] T 5f84857aad964b01b94e69af13007a55 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670
I20260924 00:20:58.862649 11156 ts_tablet_manager.cc:1905] T 5f84857aad964b01b94e69af13007a55 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260924 00:20:58.862893 10876 raft_consensus.cc:2670] T 5f84857aad964b01b94e69af13007a55 P c8626e300d0644b0908797ad61bd7cef: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260924 00:20:58.861949 11039 consensus_queue.cc:260] T 732a45a824274bdda8f7c060723a8cd2 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:58.868458 11039 ts_tablet_manager.cc:1428] T 732a45a824274bdda8f7c060723a8cd2 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.037s user 0.004s sys 0.000s
I20260924 00:20:58.869303 11039 tablet_bootstrap.cc:492] T 1877e10d1db845f2a001063fb3bffef4 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:20:58.874688 10368 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_321e6959ada84ccea6413114309bc3b8" } modify_external_catalogs: true
I20260924 00:20:58.875206 10368 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_321e6959ada84ccea6413114309bc3b8" } modify_external_catalogs: true
I20260924 00:20:58.859531 11040 tablet_bootstrap.cc:654] T b87bc7d0b65e4153aa649444fcc2c128 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:58.882853 11150 raft_consensus.cc:482] T 9a288e9b6f804283b6d48e4d4e1dd828 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:20:58.883332 11150 raft_consensus.cc:504] T 9a288e9b6f804283b6d48e4d4e1dd828 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:58.885730 11150 leader_election.cc:288] T 9a288e9b6f804283b6d48e4d4e1dd828 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
W20260924 00:20:58.888116 10732 leader_election.cc:341] T 9a288e9b6f804283b6d48e4d4e1dd828 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Not found: Tablet not found: 9a288e9b6f804283b6d48e4d4e1dd828
I20260924 00:20:58.888916 11039 tablet_bootstrap.cc:654] T 1877e10d1db845f2a001063fb3bffef4 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:58.892167 10918 tablet_service.cc:1514] Processing DeleteTablet for tablet 732a45a824274bdda8f7c060723a8cd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:56 UTC) from {username='slave'} at 127.0.0.1:35358
W20260924 00:20:58.898475 10733 leader_election.cc:341] T 9a288e9b6f804283b6d48e4d4e1dd828 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: 9a288e9b6f804283b6d48e4d4e1dd828
W20260924 00:20:58.899008 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet bcd8a6b92b8e492d96994195fa7f3ae5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bcd8a6b92b8e492d96994195fa7f3ae5 already in progress: creating tablet
I20260924 00:20:58.898867 10733 leader_election.cc:302] T 9a288e9b6f804283b6d48e4d4e1dd828 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08; no voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef
I20260924 00:20:58.899928 10632 tablet_service.cc:1514] Processing DeleteTablet for tablet b87bc7d0b65e4153aa649444fcc2c128 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:56 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:58.900489 11150 raft_consensus.cc:2726] T 9a288e9b6f804283b6d48e4d4e1dd828 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:20:58.900617 11049 tablet_replica.cc:330] T b87bc7d0b65e4153aa649444fcc2c128 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:20:58.901262 11049 raft_consensus.cc:2229] T b87bc7d0b65e4153aa649444fcc2c128 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:58.901687 11049 raft_consensus.cc:2258] T b87bc7d0b65e4153aa649444fcc2c128 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:58.903195 11049 ts_tablet_manager.cc:1905] T b87bc7d0b65e4153aa649444fcc2c128 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260924 00:20:58.916323 10592 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:35359, user_credentials={real_user=slave}} blocked reactor thread for 80167.6us
I20260924 00:20:58.917593 10592 leader_election.cc:302] T 5f84857aad964b01b94e69af13007a55 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670; no voters: c8626e300d0644b0908797ad61bd7cef
W20260924 00:20:58.918251 10592 raft_consensus.cc:2670] T 5f84857aad964b01b94e69af13007a55 P b10a9082a37b46f09d581944914c1670: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260924 00:20:58.920146 11146 raft_consensus.cc:482] T 732a45a824274bdda8f7c060723a8cd2 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:20:58.920555 11146 raft_consensus.cc:504] T 732a45a824274bdda8f7c060723a8cd2 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:58.923460 11146 leader_election.cc:288] T 732a45a824274bdda8f7c060723a8cd2 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:20:58.924199 10941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "732a45a824274bdda8f7c060723a8cd2" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:20:58.924860 10941 raft_consensus.cc:2454] T 732a45a824274bdda8f7c060723a8cd2 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b10a9082a37b46f09d581944914c1670 in term 0.
I20260924 00:20:58.924973 10800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "732a45a824274bdda8f7c060723a8cd2" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:20:58.925536 10800 raft_consensus.cc:2454] T 732a45a824274bdda8f7c060723a8cd2 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b10a9082a37b46f09d581944914c1670 in term 0.
I20260924 00:20:58.925832 10589 leader_election.cc:302] T 732a45a824274bdda8f7c060723a8cd2 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:20:58.926621 11146 raft_consensus.cc:2781] T 732a45a824274bdda8f7c060723a8cd2 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:20:58.926946 11146 raft_consensus.cc:482] T 732a45a824274bdda8f7c060723a8cd2 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:20:58.927232 11146 raft_consensus.cc:3037] T 732a45a824274bdda8f7c060723a8cd2 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:20:58.935086 11146 raft_consensus.cc:504] T 732a45a824274bdda8f7c060723a8cd2 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:58.937804 10941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "732a45a824274bdda8f7c060723a8cd2" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef"
I20260924 00:20:58.938416 10941 raft_consensus.cc:3037] T 732a45a824274bdda8f7c060723a8cd2 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:20:58.944648 11041 tablet_bootstrap.cc:492] T 6ad838c884fc4b1d8f98781e7190e3df P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:20:58.945997 11041 ts_tablet_manager.cc:1397] T 6ad838c884fc4b1d8f98781e7190e3df P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.288s user 0.010s sys 0.016s
I20260924 00:20:58.947742 11051 ts_tablet_manager.cc:1918] T 5f84857aad964b01b94e69af13007a55 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:58.948026 11041 raft_consensus.cc:348] T 6ad838c884fc4b1d8f98781e7190e3df P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:58.948632 11041 raft_consensus.cc:374] T 6ad838c884fc4b1d8f98781e7190e3df P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:58.955490 11049 ts_tablet_manager.cc:1918] T b87bc7d0b65e4153aa649444fcc2c128 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:58.955827 11049 log.cc:1196] T b87bc7d0b65e4153aa649444fcc2c128 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/b87bc7d0b65e4153aa649444fcc2c128
I20260924 00:20:58.956404 11049 ts_tablet_manager.cc:1939] T b87bc7d0b65e4153aa649444fcc2c128 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:20:58.957360 11146 leader_election.cc:288] T 732a45a824274bdda8f7c060723a8cd2 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Requested vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:20:58.960340 10365 master_service.cc:946] ReplaceTablet: received request to replace tablet 5d7d52be4f1f4fc29b8172b9fa878f76 from {username='slave'} at 127.0.0.1:47022
I20260924 00:20:58.948877 11041 raft_consensus.cc:729] T 6ad838c884fc4b1d8f98781e7190e3df P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:20:58.966053 11041 consensus_queue.cc:260] T 6ad838c884fc4b1d8f98781e7190e3df P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:58.968735 11040 tablet_bootstrap.cc:492] T b87bc7d0b65e4153aa649444fcc2c128 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:20:58.968842 10632 tablet_service.cc:1514] Processing DeleteTablet for tablet 1877e10d1db845f2a001063fb3bffef4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:56 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:58.969468 10631 tablet_service.cc:1514] Processing DeleteTablet for tablet 732a45a824274bdda8f7c060723a8cd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:56 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:58.971060 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet b87bc7d0b65e4153aa649444fcc2c128 (table default.Table_dd9f87be029b4087b002975c2067d2f3 [id=87d4d63333db40a389607571eb2171ea]) successfully deleted
I20260924 00:20:58.972584 11041 ts_tablet_manager.cc:1428] T 6ad838c884fc4b1d8f98781e7190e3df P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.026s user 0.004s sys 0.000s
I20260924 00:20:58.973384 11041 tablet_bootstrap.cc:492] T ac559183fbef4e21b190a9a61a50e1e1 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:20:58.975659 11049 tablet_replica.cc:330] T 1877e10d1db845f2a001063fb3bffef4 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:20:58.976274 11049 raft_consensus.cc:2229] T 1877e10d1db845f2a001063fb3bffef4 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:58.979357 10371 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_092022b22e2f4c378fa31f8d19f05029" } new_table_name: "default.Table_b22d604fe0dc4cad9d63baa9ffa5005c" modify_external_catalogs: true
I20260924 00:20:58.980786 11049 raft_consensus.cc:2258] T 1877e10d1db845f2a001063fb3bffef4 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:58.982964 11049 ts_tablet_manager.cc:1905] T 1877e10d1db845f2a001063fb3bffef4 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:58.987138 10365 master_service.cc:946] ReplaceTablet: received request to replace tablet c212525586544d8f9d92bc4b8bd8acaa from {username='slave'} at 127.0.0.1:47022
I20260924 00:20:58.987236 10370 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_3f0fdced96874ff3abd25f14bb25fa86" } new_table_name: "default.Table_6cf69012192f4f1087e29db70ae8c282" modify_external_catalogs: true
I20260924 00:20:59.003938 11041 tablet_bootstrap.cc:654] T ac559183fbef4e21b190a9a61a50e1e1 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:59.005517 11146 raft_consensus.cc:482] T c83942a74816465fa7f3f2c3e18c98bb P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:20:59.009039 11146 raft_consensus.cc:504] T c83942a74816465fa7f3f2c3e18c98bb P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:59.010864 11146 leader_election.cc:288] T c83942a74816465fa7f3f2c3e18c98bb P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:20:59.019567 10631 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ad838c884fc4b1d8f98781e7190e3df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:57 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:59.010373 10800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "732a45a824274bdda8f7c060723a8cd2" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:20:59.014497 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c83942a74816465fa7f3f2c3e18c98bb" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:20:59.025198 10800 raft_consensus.cc:3037] T 732a45a824274bdda8f7c060723a8cd2 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:20:59.031379 10800 raft_consensus.cc:2454] T 732a45a824274bdda8f7c060723a8cd2 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b10a9082a37b46f09d581944914c1670 in term 1.
I20260924 00:20:59.032519 11040 ts_tablet_manager.cc:1397] T b87bc7d0b65e4153aa649444fcc2c128 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.201s user 0.007s sys 0.002s
I20260924 00:20:59.035094 11040 raft_consensus.cc:348] T b87bc7d0b65e4153aa649444fcc2c128 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:59.036298 11040 raft_consensus.cc:374] T b87bc7d0b65e4153aa649444fcc2c128 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:59.037343 11041 tablet_bootstrap.cc:492] T ac559183fbef4e21b190a9a61a50e1e1 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:20:59.037724 11041 ts_tablet_manager.cc:1397] T ac559183fbef4e21b190a9a61a50e1e1 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.064s user 0.009s sys 0.000s
I20260924 00:20:59.039741 11041 raft_consensus.cc:348] T ac559183fbef4e21b190a9a61a50e1e1 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:59.040364 11041 raft_consensus.cc:374] T ac559183fbef4e21b190a9a61a50e1e1 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:59.040609 11041 raft_consensus.cc:729] T ac559183fbef4e21b190a9a61a50e1e1 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
W20260924 00:20:59.041808 10592 leader_election.cc:341] T c83942a74816465fa7f3f2c3e18c98bb P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:20:58.952600 10941 raft_consensus.cc:2454] T 732a45a824274bdda8f7c060723a8cd2 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b10a9082a37b46f09d581944914c1670 in term 1.
I20260924 00:20:58.979212 11039 tablet_bootstrap.cc:492] T 1877e10d1db845f2a001063fb3bffef4 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:20:59.042505 11039 ts_tablet_manager.cc:1397] T 1877e10d1db845f2a001063fb3bffef4 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.173s user 0.004s sys 0.020s
I20260924 00:20:59.042292 11051 log.cc:1196] T 5f84857aad964b01b94e69af13007a55 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/5f84857aad964b01b94e69af13007a55
I20260924 00:20:59.044184 11051 ts_tablet_manager.cc:1939] T 5f84857aad964b01b94e69af13007a55 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:20:59.043081 10589 leader_election.cc:302] T 732a45a824274bdda8f7c060723a8cd2 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:20:59.044863 11146 raft_consensus.cc:2781] T 732a45a824274bdda8f7c060723a8cd2 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:20:59.042043 11041 consensus_queue.cc:260] T ac559183fbef4e21b190a9a61a50e1e1 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:59.045338 11146 raft_consensus.cc:686] T 732a45a824274bdda8f7c060723a8cd2 P b10a9082a37b46f09d581944914c1670 [term 1 LEADER]: Becoming Leader. State: Replica: b10a9082a37b46f09d581944914c1670, State: Running, Role: LEADER
I20260924 00:20:59.046039 11146 consensus_queue.cc:237] T 732a45a824274bdda8f7c060723a8cd2 P b10a9082a37b46f09d581944914c1670 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:59.046613 11049 ts_tablet_manager.cc:1918] T 1877e10d1db845f2a001063fb3bffef4 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:59.046940 11049 log.cc:1196] T 1877e10d1db845f2a001063fb3bffef4 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/1877e10d1db845f2a001063fb3bffef4
I20260924 00:20:59.052907 11041 ts_tablet_manager.cc:1428] T ac559183fbef4e21b190a9a61a50e1e1 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.015s user 0.005s sys 0.000s
I20260924 00:20:59.038796 11151 raft_consensus.cc:482] T c83942a74816465fa7f3f2c3e18c98bb P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:20:59.053774 11041 tablet_bootstrap.cc:492] T 453ce17089ce44ba8da67a0893301c3c P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:20:59.053866 11151 raft_consensus.cc:504] T c83942a74816465fa7f3f2c3e18c98bb P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:59.055389 11049 ts_tablet_manager.cc:1939] T 1877e10d1db845f2a001063fb3bffef4 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
W20260924 00:20:59.036037 11169 log.cc:924] Time spent T ad256de63e0948928a51d9231152ac2d P 48fbd603abe146c585d15196e1b5be08: Append to log took a long time: real 0.070s user 0.001s sys 0.000s
I20260924 00:20:59.055702 11151 leader_election.cc:288] T c83942a74816465fa7f3f2c3e18c98bb P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:20:59.012413 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c83942a74816465fa7f3f2c3e18c98bb" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:20:59.056236 10942 raft_consensus.cc:2454] T c83942a74816465fa7f3f2c3e18c98bb P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b10a9082a37b46f09d581944914c1670 in term 0.
I20260924 00:20:59.057550 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 1877e10d1db845f2a001063fb3bffef4 (table default.Table_5f8c295237ab4e0ea184a03a9fa458a2 [id=be698d8b82734ff5940f6915ba1d4df1]) successfully deleted
I20260924 00:20:59.058089 10799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c83942a74816465fa7f3f2c3e18c98bb" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:20:59.058429 10655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c83942a74816465fa7f3f2c3e18c98bb" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:20:59.058988 10655 raft_consensus.cc:2454] T c83942a74816465fa7f3f2c3e18c98bb P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
W20260924 00:20:59.059201 10878 leader_election.cc:341] T c83942a74816465fa7f3f2c3e18c98bb P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:20:59.060650 11040 raft_consensus.cc:729] T b87bc7d0b65e4153aa649444fcc2c128 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:20:59.060523 10589 leader_election.cc:302] T c83942a74816465fa7f3f2c3e18c98bb P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08
I20260924 00:20:59.062433 10876 leader_election.cc:302] T c83942a74816465fa7f3f2c3e18c98bb P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08
I20260924 00:20:59.045231 11039 raft_consensus.cc:348] T 1877e10d1db845f2a001063fb3bffef4 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:59.063457 11039 raft_consensus.cc:374] T 1877e10d1db845f2a001063fb3bffef4 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:59.063706 11039 raft_consensus.cc:729] T 1877e10d1db845f2a001063fb3bffef4 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:20:59.064246 11039 consensus_queue.cc:260] T 1877e10d1db845f2a001063fb3bffef4 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:59.066089 11151 raft_consensus.cc:2781] T c83942a74816465fa7f3f2c3e18c98bb P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:20:59.066387 11151 raft_consensus.cc:482] T c83942a74816465fa7f3f2c3e18c98bb P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:20:59.066669 11151 raft_consensus.cc:3037] T c83942a74816465fa7f3f2c3e18c98bb P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:20:59.071039 10776 tablet_service.cc:1467] Processing CreateTablet for tablet 0980afc2af174da2a8ca2dd414e9825c (DEFAULT_TABLE table=default.Table_e83ceabd11404e14bb1eee103fffaaed [id=222d2010881c4b5c9635aa3c28adc72b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:59.074038 11146 raft_consensus.cc:2781] T c83942a74816465fa7f3f2c3e18c98bb P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:20:59.074366 11146 raft_consensus.cc:482] T c83942a74816465fa7f3f2c3e18c98bb P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:20:59.074662 11146 raft_consensus.cc:3037] T c83942a74816465fa7f3f2c3e18c98bb P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:20:59.075402 10918 tablet_service.cc:1467] Processing CreateTablet for tablet 0980afc2af174da2a8ca2dd414e9825c (DEFAULT_TABLE table=default.Table_e83ceabd11404e14bb1eee103fffaaed [id=222d2010881c4b5c9635aa3c28adc72b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:59.076800 10918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0980afc2af174da2a8ca2dd414e9825c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:59.077780 10774 tablet_service.cc:1467] Processing CreateTablet for tablet 6ac9c7da7fb546f6ae935371799cbad6 (DEFAULT_TABLE table=default.Table_092022b22e2f4c378fa31f8d19f05029 [id=9aee3bf701b640cfb7bcba395d4225b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:59.080399 10632 tablet_service.cc:1467] Processing CreateTablet for tablet 6ac9c7da7fb546f6ae935371799cbad6 (DEFAULT_TABLE table=default.Table_092022b22e2f4c378fa31f8d19f05029 [id=9aee3bf701b640cfb7bcba395d4225b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:59.081537 10632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ac9c7da7fb546f6ae935371799cbad6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:59.082115 10920 tablet_service.cc:1467] Processing CreateTablet for tablet 6ac9c7da7fb546f6ae935371799cbad6 (DEFAULT_TABLE table=default.Table_092022b22e2f4c378fa31f8d19f05029 [id=9aee3bf701b640cfb7bcba395d4225b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:59.083068 10920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ac9c7da7fb546f6ae935371799cbad6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:59.085536 10634 tablet_service.cc:1467] Processing CreateTablet for tablet 195b033091b543b58ee08417eae2c0e6 (DEFAULT_TABLE table=default.Table_ca083375f0df4255b8654ed88040dfcd [id=f79ca8d11aa04d5f8f72d58b798ed4ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:59.086514 10634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 195b033091b543b58ee08417eae2c0e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:59.086220 10369 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_95d86f2882084cb6821c2aa52497b554"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:20:59.088435 10369 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_95d86f2882084cb6821c2aa52497b554 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:59.089864 10775 tablet_service.cc:1467] Processing CreateTablet for tablet 195b033091b543b58ee08417eae2c0e6 (DEFAULT_TABLE table=default.Table_ca083375f0df4255b8654ed88040dfcd [id=f79ca8d11aa04d5f8f72d58b798ed4ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:59.091259 10780 tablet_service.cc:1467] Processing CreateTablet for tablet 772fb3d6180e499daf94ceeb9b26048d (DEFAULT_TABLE table=default.Table_9c2aa717e62340c785be379c0dd5837c [id=737634d9f0f4413c8b6dcaa26de87e2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:59.093219 10633 tablet_service.cc:1467] Processing CreateTablet for tablet 772fb3d6180e499daf94ceeb9b26048d (DEFAULT_TABLE table=default.Table_9c2aa717e62340c785be379c0dd5837c [id=737634d9f0f4413c8b6dcaa26de87e2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:59.093524 11049 tablet_replica.cc:330] T 732a45a824274bdda8f7c060723a8cd2 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:20:59.093487 10916 tablet_service.cc:1467] Processing CreateTablet for tablet 195b033091b543b58ee08417eae2c0e6 (DEFAULT_TABLE table=default.Table_ca083375f0df4255b8654ed88040dfcd [id=f79ca8d11aa04d5f8f72d58b798ed4ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:59.094450 10916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 195b033091b543b58ee08417eae2c0e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:59.101007 10921 tablet_service.cc:1467] Processing CreateTablet for tablet 772fb3d6180e499daf94ceeb9b26048d (DEFAULT_TABLE table=default.Table_9c2aa717e62340c785be379c0dd5837c [id=737634d9f0f4413c8b6dcaa26de87e2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:59.102144 10921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 772fb3d6180e499daf94ceeb9b26048d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:59.103510 10778 tablet_service.cc:1467] Processing CreateTablet for tablet f77aea0aad5a430f941aaec0a1c19640 (DEFAULT_TABLE table=default.Table_87fc5b8981644a5d862af47154aaa22a [id=da26c922964143e789bc49d07d8eb880]), partition=RANGE (key) PARTITION UNBOUNDED
W20260924 00:20:59.108891 10370 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ffa011cf8dc446de849bea75a4f20056 has no consensus state
I20260924 00:20:59.112701 10917 tablet_service.cc:1467] Processing CreateTablet for tablet f77aea0aad5a430f941aaec0a1c19640 (DEFAULT_TABLE table=default.Table_87fc5b8981644a5d862af47154aaa22a [id=da26c922964143e789bc49d07d8eb880]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:59.113972 10917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f77aea0aad5a430f941aaec0a1c19640. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:59.114405 11151 raft_consensus.cc:504] T c83942a74816465fa7f3f2c3e18c98bb P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:59.121376 10370 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_cca0dd1d3cd247398f21a23a2724f3fc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:20:59.123106 10370 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cca0dd1d3cd247398f21a23a2724f3fc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:59.094172 11049 raft_consensus.cc:2229] T 732a45a824274bdda8f7c060723a8cd2 P b10a9082a37b46f09d581944914c1670 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:20:59.126446 11049 pending_rounds.cc:62] T 732a45a824274bdda8f7c060723a8cd2 P b10a9082a37b46f09d581944914c1670: Trying to abort 1 pending ops.
I20260924 00:20:59.116865 10655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c83942a74816465fa7f3f2c3e18c98bb" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:20:59.094180 10633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 772fb3d6180e499daf94ceeb9b26048d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:59.127810 10631 tablet_service.cc:1467] Processing CreateTablet for tablet 0980afc2af174da2a8ca2dd414e9825c (DEFAULT_TABLE table=default.Table_e83ceabd11404e14bb1eee103fffaaed [id=222d2010881c4b5c9635aa3c28adc72b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:59.113313 10636 tablet_service.cc:1467] Processing CreateTablet for tablet f77aea0aad5a430f941aaec0a1c19640 (DEFAULT_TABLE table=default.Table_87fc5b8981644a5d862af47154aaa22a [id=da26c922964143e789bc49d07d8eb880]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:59.130033 10636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f77aea0aad5a430f941aaec0a1c19640. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:59.137580 11040 consensus_queue.cc:260] T b87bc7d0b65e4153aa649444fcc2c128 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:59.138298 11041 tablet_bootstrap.cc:654] T 453ce17089ce44ba8da67a0893301c3c P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
W20260924 00:20:59.137863 10371 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6ac9c7da7fb546f6ae935371799cbad6 consensus state has no leader
I20260924 00:20:59.126681 11049 pending_rounds.cc:68] T 732a45a824274bdda8f7c060723a8cd2 P b10a9082a37b46f09d581944914c1670: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332697125060743168 op_type: NO_OP noop_request { }
I20260924 00:20:59.140690 11049 raft_consensus.cc:2866] T 732a45a824274bdda8f7c060723a8cd2 P b10a9082a37b46f09d581944914c1670 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260924 00:20:59.141132 11049 raft_consensus.cc:2258] T 732a45a824274bdda8f7c060723a8cd2 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:59.142246 11146 raft_consensus.cc:504] T c83942a74816465fa7f3f2c3e18c98bb P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:59.143338 10655 raft_consensus.cc:2379] T c83942a74816465fa7f3f2c3e18c98bb P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c8626e300d0644b0908797ad61bd7cef in current term 1: Already voted for candidate b10a9082a37b46f09d581944914c1670 in this term.
I20260924 00:20:59.144165 10631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0980afc2af174da2a8ca2dd414e9825c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:59.146266 11049 ts_tablet_manager.cc:1905] T 732a45a824274bdda8f7c060723a8cd2 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:59.150914 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ac9c7da7fb546f6ae935371799cbad6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:59.152016 10776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0980afc2af174da2a8ca2dd414e9825c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:59.157178 10780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 772fb3d6180e499daf94ceeb9b26048d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:59.159837 10773 tablet_service.cc:1514] Processing DeleteTablet for tablet b87bc7d0b65e4153aa649444fcc2c128 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:56 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:20:59.161798 10778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f77aea0aad5a430f941aaec0a1c19640. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:59.166100 10775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 195b033091b543b58ee08417eae2c0e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:59.176807 10374 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 0d555f059f84478eb805f4ca4534dab4
I20260924 00:20:59.178580 11146 leader_election.cc:288] T c83942a74816465fa7f3f2c3e18c98bb P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Requested vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:20:59.197113 10374 master_service.cc:946] ReplaceTablet: received request to replace tablet d44932f8f08e4ea7acc05b8ec511f045 from {username='slave'} at 127.0.0.1:47022
I20260924 00:20:59.215299 11041 tablet_bootstrap.cc:492] T 453ce17089ce44ba8da67a0893301c3c P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:20:59.215706 11041 ts_tablet_manager.cc:1397] T 453ce17089ce44ba8da67a0893301c3c P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.162s user 0.006s sys 0.003s
I20260924 00:20:59.219782 11041 raft_consensus.cc:348] T 453ce17089ce44ba8da67a0893301c3c P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:59.220433 11041 raft_consensus.cc:374] T 453ce17089ce44ba8da67a0893301c3c P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:59.220678 11041 raft_consensus.cc:729] T 453ce17089ce44ba8da67a0893301c3c P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:20:59.226589 10373 master_service.cc:946] ReplaceTablet: received request to replace tablet eeb11092b803467e9db732468350e852 from {username='slave'} at 127.0.0.1:47022
I20260924 00:20:59.240603 11049 ts_tablet_manager.cc:1918] T 732a45a824274bdda8f7c060723a8cd2 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:20:59.248404 10368 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet d44932f8f08e4ea7acc05b8ec511f045
I20260924 00:20:59.250034 10371 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_a7b8969040dd4ef5ad547c617f03d471"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:20:59.251674 10371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a7b8969040dd4ef5ad547c617f03d471 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:59.251859 11051 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35358 (request call id 54) took 1539 ms. Trace:
I20260924 00:20:59.256562 11040 ts_tablet_manager.cc:1428] T b87bc7d0b65e4153aa649444fcc2c128 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.224s user 0.005s sys 0.000s
I20260924 00:20:59.261579 10776 tablet_service.cc:1514] Processing DeleteTablet for tablet d44932f8f08e4ea7acc05b8ec511f045 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:58 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:20:59.272709 10370 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_f1bc26fec8e746ba84ca80afe98a4d26"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:20:59.273010 11051 rpcz_store.cc:276] 0924 00:20:57.712757 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:20:57.712877 (+ 120us) service_pool.cc:227] Handling call
0924 00:20:58.468896 (+756019us) ts_tablet_manager.cc:1145] Deleting tablet 5f84857aad964b01b94e69af13007a55
0924 00:20:58.468934 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:20:58.747865 (+278931us) tablet_metadata.cc:641] Metadata flushed
0924 00:20:58.947548 (+199683us) tablet_metadata.cc:641] Metadata flushed
0924 00:20:59.251785 (+304237us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":755452}
I20260924 00:20:59.274757 11051 tablet_replica.cc:330] T b87bc7d0b65e4153aa649444fcc2c128 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:20:59.275520 11051 raft_consensus.cc:2229] T b87bc7d0b65e4153aa649444fcc2c128 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Raft consensus shutting down.
W20260924 00:20:59.281774 10370 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f1bc26fec8e746ba84ca80afe98a4d26 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:59.251746 10633 tablet_service.cc:1514] Processing DeleteTablet for tablet d44932f8f08e4ea7acc05b8ec511f045 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:58 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:59.283118 11049 log.cc:1196] T 732a45a824274bdda8f7c060723a8cd2 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/732a45a824274bdda8f7c060723a8cd2
I20260924 00:20:59.284231 11049 ts_tablet_manager.cc:1939] T 732a45a824274bdda8f7c060723a8cd2 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:20:59.285992 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 5f84857aad964b01b94e69af13007a55 (table default.Table_d4b54eed2b8a4b3994b2635e01c5c273 [id=143547db8d2d4517877751965d3042ce]) successfully deleted
I20260924 00:20:59.286845 11049 tablet_replica.cc:330] T 6ad838c884fc4b1d8f98781e7190e3df P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:20:59.287014 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 732a45a824274bdda8f7c060723a8cd2 (table default.Table_ce92a0828938401292681552add70c45 [id=18a2080b0c7344a28df81315ee1e5cc3]) successfully deleted
I20260924 00:20:59.287467 11049 raft_consensus.cc:2229] T 6ad838c884fc4b1d8f98781e7190e3df P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:59.287878 11049 raft_consensus.cc:2258] T 6ad838c884fc4b1d8f98781e7190e3df P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:59.261509 11041 consensus_queue.cc:260] T 453ce17089ce44ba8da67a0893301c3c P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:59.290205 11041 ts_tablet_manager.cc:1428] T 453ce17089ce44ba8da67a0893301c3c P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.074s user 0.010s sys 0.001s
I20260924 00:20:59.291024 11041 tablet_bootstrap.cc:492] T e0074b1184ec4883ab1016330d98c968 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:20:59.291895 10799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c83942a74816465fa7f3f2c3e18c98bb" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
W20260924 00:20:59.295051 10592 leader_election.cc:341] T c83942a74816465fa7f3f2c3e18c98bb P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:20:59.296185 11049 ts_tablet_manager.cc:1905] T 6ad838c884fc4b1d8f98781e7190e3df P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:59.306270 11041 tablet_bootstrap.cc:654] T e0074b1184ec4883ab1016330d98c968 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:59.315634 11049 ts_tablet_manager.cc:1918] T 6ad838c884fc4b1d8f98781e7190e3df P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:59.315984 11049 log.cc:1196] T 6ad838c884fc4b1d8f98781e7190e3df P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/6ad838c884fc4b1d8f98781e7190e3df
I20260924 00:20:59.316531 11049 ts_tablet_manager.cc:1939] T 6ad838c884fc4b1d8f98781e7190e3df P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:20:59.318773 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 6ad838c884fc4b1d8f98781e7190e3df (table default.Table_2d4af2cf18544b6e94ef78ac7dd3bcce [id=218c77bc322a488593d12af9a27d20a3]) successfully deleted
I20260924 00:20:59.322332 10633 tablet_service.cc:1514] Processing DeleteTablet for tablet 732a45a824274bdda8f7c060723a8cd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:56 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:59.322886 10636 tablet_service.cc:1514] Processing DeleteTablet for tablet ac559183fbef4e21b190a9a61a50e1e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:56 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:59.329808 11041 tablet_bootstrap.cc:492] T e0074b1184ec4883ab1016330d98c968 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:20:59.330331 11041 ts_tablet_manager.cc:1397] T e0074b1184ec4883ab1016330d98c968 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.039s user 0.007s sys 0.008s
I20260924 00:20:59.332468 11041 raft_consensus.cc:348] T e0074b1184ec4883ab1016330d98c968 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
W20260924 00:20:59.334203 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet d44932f8f08e4ea7acc05b8ec511f045 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d44932f8f08e4ea7acc05b8ec511f045 already in progress: creating tablet
I20260924 00:20:59.334818 11041 raft_consensus.cc:374] T e0074b1184ec4883ab1016330d98c968 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:59.335168 11041 raft_consensus.cc:729] T e0074b1184ec4883ab1016330d98c968 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
W20260924 00:20:59.336440 10359 catalog_manager.cc:4808] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 732a45a824274bdda8f7c060723a8cd2 because the tablet was not found. No further retry: Not found: Tablet not found: 732a45a824274bdda8f7c060723a8cd2
I20260924 00:20:59.286275 11039 ts_tablet_manager.cc:1428] T 1877e10d1db845f2a001063fb3bffef4 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.244s user 0.035s sys 0.036s
I20260924 00:20:59.336506 11041 consensus_queue.cc:260] T e0074b1184ec4883ab1016330d98c968 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:59.337646 11039 tablet_bootstrap.cc:492] T 6ad838c884fc4b1d8f98781e7190e3df P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:20:59.337669 11049 tablet_replica.cc:330] T ac559183fbef4e21b190a9a61a50e1e1 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:20:59.286681 10919 tablet_service.cc:1514] Processing DeleteTablet for tablet d44932f8f08e4ea7acc05b8ec511f045 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:58 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:20:59.339563 11049 raft_consensus.cc:2229] T ac559183fbef4e21b190a9a61a50e1e1 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:59.340749 11051 pending_rounds.cc:62] T b87bc7d0b65e4153aa649444fcc2c128 P c8626e300d0644b0908797ad61bd7cef: Trying to abort 1 pending ops.
I20260924 00:20:59.341216 11049 raft_consensus.cc:2258] T ac559183fbef4e21b190a9a61a50e1e1 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:59.341560 11041 ts_tablet_manager.cc:1428] T e0074b1184ec4883ab1016330d98c968 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.011s user 0.004s sys 0.001s
W20260924 00:20:59.339502 10592 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:35359, user_credentials={real_user=slave}} blocked reactor thread for 44489.5us
I20260924 00:20:59.314536 10371 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_b8d64d06746d4e1f8f357136d1cebfe9" } new_table_name: "default.Table_67493ad72aac4bbca26a6a322a60e5cd" modify_external_catalogs: true
I20260924 00:20:59.343474 11041 tablet_bootstrap.cc:492] T ccb26e0fbc1948bc9a00e9e26c4d1139 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:20:59.347924 10373 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet eeb11092b803467e9db732468350e852
I20260924 00:20:59.348273 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c83942a74816465fa7f3f2c3e18c98bb" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef"
I20260924 00:20:59.349247 10373 catalog_manager.cc:6565] ReplaceTablet: tablet eeb11092b803467e9db732468350e852 deleted and replaced by tablet 4325b60efc1649c9ba4828d0a098a676
I20260924 00:20:59.349408 11049 ts_tablet_manager.cc:1905] T ac559183fbef4e21b190a9a61a50e1e1 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:59.350235 10366 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_e83ceabd11404e14bb1eee103fffaaed" } new_table_name: "default.Table_0adf8e54351d4721846a55224f2831d5" modify_external_catalogs: true
I20260924 00:20:59.353860 11040 tablet_bootstrap.cc:492] T c83942a74816465fa7f3f2c3e18c98bb P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:20:59.355648 10636 tablet_service.cc:1514] Processing DeleteTablet for tablet eeb11092b803467e9db732468350e852 with delete_type TABLET_DATA_DELETED (replaced by tablet 4325b60efc1649c9ba4828d0a098a676) from {username='slave'} at 127.0.0.1:46896
W20260924 00:20:59.288287 10876 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:33461, user_credentials={real_user=slave}} blocked reactor thread for 82845.7us
I20260924 00:20:59.341099 11051 pending_rounds.cc:68] T b87bc7d0b65e4153aa649444fcc2c128 P c8626e300d0644b0908797ad61bd7cef: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332697122690146304 op_type: NO_OP noop_request { }
I20260924 00:20:59.359337 10942 raft_consensus.cc:2379] T c83942a74816465fa7f3f2c3e18c98bb P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b10a9082a37b46f09d581944914c1670 in current term 1: Already voted for candidate c8626e300d0644b0908797ad61bd7cef in this term.
I20260924 00:20:59.359791 11051 raft_consensus.cc:2866] T b87bc7d0b65e4153aa649444fcc2c128 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260924 00:20:59.361032 10589 leader_election.cc:302] T c83942a74816465fa7f3f2c3e18c98bb P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:20:59.362051 11146 raft_consensus.cc:2726] T c83942a74816465fa7f3f2c3e18c98bb P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260924 00:20:59.365788 10799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c83942a74816465fa7f3f2c3e18c98bb" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:20:59.340653 11151 leader_election.cc:288] T c83942a74816465fa7f3f2c3e18c98bb P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:20:59.370548 10778 tablet_service.cc:1514] Processing DeleteTablet for tablet eeb11092b803467e9db732468350e852 with delete_type TABLET_DATA_DELETED (replaced by tablet 4325b60efc1649c9ba4828d0a098a676) from {username='slave'} at 127.0.0.1:35996
I20260924 00:20:59.371840 11041 tablet_bootstrap.cc:654] T ccb26e0fbc1948bc9a00e9e26c4d1139 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:59.363731 11039 tablet_bootstrap.cc:654] T 6ad838c884fc4b1d8f98781e7190e3df P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:59.366569 10919 tablet_service.cc:1514] Processing DeleteTablet for tablet eeb11092b803467e9db732468350e852 with delete_type TABLET_DATA_DELETED (replaced by tablet 4325b60efc1649c9ba4828d0a098a676) from {username='slave'} at 127.0.0.1:35358
W20260924 00:20:59.368121 10878 leader_election.cc:341] T c83942a74816465fa7f3f2c3e18c98bb P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:20:59.369747 11051 raft_consensus.cc:2258] T b87bc7d0b65e4153aa649444fcc2c128 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:59.378731 11051 ts_tablet_manager.cc:1905] T b87bc7d0b65e4153aa649444fcc2c128 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:59.375623 10878 leader_election.cc:302] T c83942a74816465fa7f3f2c3e18c98bb P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670
I20260924 00:20:59.386116 11049 ts_tablet_manager.cc:1918] T ac559183fbef4e21b190a9a61a50e1e1 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:59.386479 11049 log.cc:1196] T ac559183fbef4e21b190a9a61a50e1e1 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/ac559183fbef4e21b190a9a61a50e1e1
I20260924 00:20:59.386965 11049 ts_tablet_manager.cc:1939] T ac559183fbef4e21b190a9a61a50e1e1 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:20:59.388919 11151 raft_consensus.cc:2726] T c83942a74816465fa7f3f2c3e18c98bb P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260924 00:20:59.389087 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet ac559183fbef4e21b190a9a61a50e1e1 (table default.Table_3b2077a1a0a446f98a6ddb235c703345 [id=8063a759be2141ae965703f9196a18c8]) successfully deleted
I20260924 00:20:59.392504 11041 tablet_bootstrap.cc:492] T ccb26e0fbc1948bc9a00e9e26c4d1139 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:20:59.392911 11041 ts_tablet_manager.cc:1397] T ccb26e0fbc1948bc9a00e9e26c4d1139 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.050s user 0.006s sys 0.004s
I20260924 00:20:59.396062 11041 raft_consensus.cc:348] T ccb26e0fbc1948bc9a00e9e26c4d1139 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:59.396724 11041 raft_consensus.cc:374] T ccb26e0fbc1948bc9a00e9e26c4d1139 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:59.397014 11041 raft_consensus.cc:729] T ccb26e0fbc1948bc9a00e9e26c4d1139 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:20:59.397718 11041 consensus_queue.cc:260] T ccb26e0fbc1948bc9a00e9e26c4d1139 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
W20260924 00:20:59.399526 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet eeb11092b803467e9db732468350e852 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet eeb11092b803467e9db732468350e852 already in progress: creating tablet
I20260924 00:20:59.403237 11041 ts_tablet_manager.cc:1428] T ccb26e0fbc1948bc9a00e9e26c4d1139 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20260924 00:20:59.405275 11041 tablet_bootstrap.cc:492] T af2f58e6757a4b738a8ec38b2e228f9f P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:20:59.412277 11041 tablet_bootstrap.cc:654] T af2f58e6757a4b738a8ec38b2e228f9f P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:59.413568 11146 raft_consensus.cc:482] T 453ce17089ce44ba8da67a0893301c3c P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:20:59.413996 11146 raft_consensus.cc:504] T 453ce17089ce44ba8da67a0893301c3c P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:59.415848 11146 leader_election.cc:288] T 453ce17089ce44ba8da67a0893301c3c P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
W20260924 00:20:59.418195 10366 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0980afc2af174da2a8ca2dd414e9825c consensus state has no leader
I20260924 00:20:59.418555 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "453ce17089ce44ba8da67a0893301c3c" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:20:59.419451 10799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "453ce17089ce44ba8da67a0893301c3c" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:20:59.419967 10589 leader_election.cc:341] T 453ce17089ce44ba8da67a0893301c3c P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:20:59.420089 11156 ts_tablet_manager.cc:1918] T 5f84857aad964b01b94e69af13007a55 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:59.421104 10919 tablet_service.cc:1514] Processing DeleteTablet for tablet 1877e10d1db845f2a001063fb3bffef4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:56 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:20:59.421211 10374 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_89e5d14641754be1934c7e3c880b21ee" } modify_external_catalogs: true
I20260924 00:20:59.421756 10374 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_89e5d14641754be1934c7e3c880b21ee" } modify_external_catalogs: true
W20260924 00:20:59.421937 10592 leader_election.cc:341] T 453ce17089ce44ba8da67a0893301c3c P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:20:59.422292 10592 leader_election.cc:302] T 453ce17089ce44ba8da67a0893301c3c P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:20:59.423471 11146 raft_consensus.cc:2726] T 453ce17089ce44ba8da67a0893301c3c P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260924 00:20:59.429958 10360 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet ffa011cf8dc446de849bea75a4f20056 (table default.Table_6cf69012192f4f1087e29db70ae8c282 [id=93c1734de58245f8b1d5ab88ec9f0c8e], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet ffa011cf8dc446de849bea75a4f20056 has no consensus state
W20260924 00:20:59.425786 10371 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 85a0ad61b7cb4e11b30e84ddc4655b64 consensus state has no leader
I20260924 00:20:59.432096 11041 tablet_bootstrap.cc:492] T af2f58e6757a4b738a8ec38b2e228f9f P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:20:59.432489 11041 ts_tablet_manager.cc:1397] T af2f58e6757a4b738a8ec38b2e228f9f P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.027s user 0.004s sys 0.009s
I20260924 00:20:59.434540 11041 raft_consensus.cc:348] T af2f58e6757a4b738a8ec38b2e228f9f P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:20:59.435159 11041 raft_consensus.cc:374] T af2f58e6757a4b738a8ec38b2e228f9f P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:59.435405 11041 raft_consensus.cc:729] T af2f58e6757a4b738a8ec38b2e228f9f P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:20:59.436184 11041 consensus_queue.cc:260] T af2f58e6757a4b738a8ec38b2e228f9f P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:20:59.439193 11041 ts_tablet_manager.cc:1428] T af2f58e6757a4b738a8ec38b2e228f9f P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.006s user 0.000s sys 0.006s
I20260924 00:20:59.420444 11156 log.cc:1196] T 5f84857aad964b01b94e69af13007a55 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/5f84857aad964b01b94e69af13007a55
I20260924 00:20:59.440220 11156 ts_tablet_manager.cc:1939] T 5f84857aad964b01b94e69af13007a55 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:20:59.444299 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 5f84857aad964b01b94e69af13007a55 (table default.Table_d4b54eed2b8a4b3994b2635e01c5c273 [id=143547db8d2d4517877751965d3042ce]) successfully deleted
I20260924 00:20:59.444691 11041 tablet_bootstrap.cc:492] T c212525586544d8f9d92bc4b8bd8acaa P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:20:59.450807 11156 tablet_replica.cc:330] T 9a288e9b6f804283b6d48e4d4e1dd828 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:20:59.451539 11156 raft_consensus.cc:2229] T 9a288e9b6f804283b6d48e4d4e1dd828 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:59.451992 11156 raft_consensus.cc:2258] T 9a288e9b6f804283b6d48e4d4e1dd828 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:59.453830 11156 ts_tablet_manager.cc:1905] T 9a288e9b6f804283b6d48e4d4e1dd828 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:59.456997 11040 tablet_bootstrap.cc:654] T c83942a74816465fa7f3f2c3e18c98bb P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:59.461378 10778 tablet_service.cc:1467] Processing CreateTablet for tablet b06bf1e4691441769e162aa5404d4656 (DEFAULT_TABLE table=default.Table_6a2c714f26254ca89f7021633fc89fb7 [id=534f5d0c2e6e426e834efb9b089e4651]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:59.462261 10778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b06bf1e4691441769e162aa5404d4656. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:59.461099 10920 tablet_service.cc:1467] Processing CreateTablet for tablet ffa011cf8dc446de849bea75a4f20056 (DEFAULT_TABLE table=default.Table_6cf69012192f4f1087e29db70ae8c282 [id=93c1734de58245f8b1d5ab88ec9f0c8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:59.463163 10920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffa011cf8dc446de849bea75a4f20056. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:59.464895 10366 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_6a2c714f26254ca89f7021633fc89fb7" } modify_external_catalogs: true
I20260924 00:20:59.465056 10370 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_e86eeb97d87d4f9ea7edfe269cc1d790"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:20:59.465461 10366 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_6a2c714f26254ca89f7021633fc89fb7" } modify_external_catalogs: true
I20260924 00:20:59.465642 10921 tablet_service.cc:1467] Processing CreateTablet for tablet b06bf1e4691441769e162aa5404d4656 (DEFAULT_TABLE table=default.Table_6a2c714f26254ca89f7021633fc89fb7 [id=534f5d0c2e6e426e834efb9b089e4651]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:59.466625 10921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b06bf1e4691441769e162aa5404d4656. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:20:59.466856 10370 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e86eeb97d87d4f9ea7edfe269cc1d790 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:59.468535 11156 ts_tablet_manager.cc:1918] T 9a288e9b6f804283b6d48e4d4e1dd828 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:59.468911 11156 log.cc:1196] T 9a288e9b6f804283b6d48e4d4e1dd828 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/9a288e9b6f804283b6d48e4d4e1dd828
I20260924 00:20:59.469556 11156 ts_tablet_manager.cc:1939] T 9a288e9b6f804283b6d48e4d4e1dd828 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:20:59.471506 10636 tablet_service.cc:1467] Processing CreateTablet for tablet b06bf1e4691441769e162aa5404d4656 (DEFAULT_TABLE table=default.Table_6a2c714f26254ca89f7021633fc89fb7 [id=534f5d0c2e6e426e834efb9b089e4651]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:59.471750 10373 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet e0074b1184ec4883ab1016330d98c968 (table default.Table_a54d4fd8353e42a4a00cb4e5c689da7b [id=485c42c942b74abbb32bea36d25bc184]). Expected version 1 got 0
I20260924 00:20:59.472826 10633 tablet_service.cc:1467] Processing CreateTablet for tablet ffa011cf8dc446de849bea75a4f20056 (DEFAULT_TABLE table=default.Table_6cf69012192f4f1087e29db70ae8c282 [id=93c1734de58245f8b1d5ab88ec9f0c8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:59.474166 10632 tablet_service.cc:1467] Processing CreateTablet for tablet 0d555f059f84478eb805f4ca4534dab4 (DEFAULT_TABLE table=default.Table_99da18207fc542fbb68972724c4d410e [id=f75c1da3bd734e92bc105c473236fa9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:59.477000 10634 tablet_service.cc:1514] Processing DeleteTablet for tablet ac559183fbef4e21b190a9a61a50e1e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:56 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:59.477363 10635 tablet_service.cc:1514] Processing DeleteTablet for tablet 453ce17089ce44ba8da67a0893301c3c with delete_type TABLET_DATA_DELETED (replaced by tablet 195b033091b543b58ee08417eae2c0e6) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:59.478109 11049 tablet_replica.cc:330] T 453ce17089ce44ba8da67a0893301c3c P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:20:59.478739 11049 raft_consensus.cc:2229] T 453ce17089ce44ba8da67a0893301c3c P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260924 00:20:59.478919 10359 catalog_manager.cc:4808] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet ac559183fbef4e21b190a9a61a50e1e1 because the tablet was not found. No further retry: Not found: Tablet not found: ac559183fbef4e21b190a9a61a50e1e1
I20260924 00:20:59.479146 11049 raft_consensus.cc:2258] T 453ce17089ce44ba8da67a0893301c3c P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:59.481220 11049 ts_tablet_manager.cc:1905] T 453ce17089ce44ba8da67a0893301c3c P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:59.479040 10917 tablet_service.cc:1467] Processing CreateTablet for tablet 0d555f059f84478eb805f4ca4534dab4 (DEFAULT_TABLE table=default.Table_99da18207fc542fbb68972724c4d410e [id=f75c1da3bd734e92bc105c473236fa9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:59.485764 10633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffa011cf8dc446de849bea75a4f20056. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:59.486172 10917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d555f059f84478eb805f4ca4534dab4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:59.490545 11039 tablet_bootstrap.cc:492] T 6ad838c884fc4b1d8f98781e7190e3df P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:20:59.490985 11039 ts_tablet_manager.cc:1397] T 6ad838c884fc4b1d8f98781e7190e3df P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.153s user 0.010s sys 0.003s
I20260924 00:20:59.492031 10636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b06bf1e4691441769e162aa5404d4656. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:59.495568 10632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d555f059f84478eb805f4ca4534dab4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:59.497660 10780 tablet_service.cc:1467] Processing CreateTablet for tablet c7e50ac8b7f947b1bfba8cd1cff0ea7e (DEFAULT_TABLE table=default.Table_593b4d43f8144c04ab048e8221b3fc8e [id=951c867d04f74728a684d20d1104c054]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:59.498598 10780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7e50ac8b7f947b1bfba8cd1cff0ea7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:59.503886 10918 tablet_service.cc:1467] Processing CreateTablet for tablet c7e50ac8b7f947b1bfba8cd1cff0ea7e (DEFAULT_TABLE table=default.Table_593b4d43f8144c04ab048e8221b3fc8e [id=951c867d04f74728a684d20d1104c054]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:59.505228 10918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7e50ac8b7f947b1bfba8cd1cff0ea7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:59.505002 10776 tablet_service.cc:1467] Processing CreateTablet for tablet 119d83562ef546cab496cd7b44fecc15 (DEFAULT_TABLE table=default.Table_a6631f3ce9a04312b19aa2e85b4c9fe3 [id=5fb5e1e90c9440f385e7b04520bcba9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:59.505950 10776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 119d83562ef546cab496cd7b44fecc15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:59.507169 10635 tablet_service.cc:1467] Processing CreateTablet for tablet c7e50ac8b7f947b1bfba8cd1cff0ea7e (DEFAULT_TABLE table=default.Table_593b4d43f8144c04ab048e8221b3fc8e [id=951c867d04f74728a684d20d1104c054]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:59.508049 10635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7e50ac8b7f947b1bfba8cd1cff0ea7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:59.512053 11041 tablet_bootstrap.cc:654] T c212525586544d8f9d92bc4b8bd8acaa P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:59.513286 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 9a288e9b6f804283b6d48e4d4e1dd828 (table default.Table_5f8c295237ab4e0ea184a03a9fa458a2 [id=be698d8b82734ff5940f6915ba1d4df1]) successfully deleted
I20260924 00:20:59.514156 10368 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_ff66b5b591e74da99ad87f8512cdc082"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:20:59.515436 11156 tablet_replica.cc:330] T 732a45a824274bdda8f7c060723a8cd2 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
W20260924 00:20:59.515728 10368 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ff66b5b591e74da99ad87f8512cdc082 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:59.516139 11156 raft_consensus.cc:2229] T 732a45a824274bdda8f7c060723a8cd2 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:59.516552 11156 raft_consensus.cc:2258] T 732a45a824274bdda8f7c060723a8cd2 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:59.517401 10371 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_9f2df24361da4f9a9340f6f8673043ea"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:20:59.518638 11156 ts_tablet_manager.cc:1905] T 732a45a824274bdda8f7c060723a8cd2 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:59.520537 10774 tablet_service.cc:1467] Processing CreateTablet for tablet ffa011cf8dc446de849bea75a4f20056 (DEFAULT_TABLE table=default.Table_6cf69012192f4f1087e29db70ae8c282 [id=93c1734de58245f8b1d5ab88ec9f0c8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:59.521555 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffa011cf8dc446de849bea75a4f20056. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:59.525750 10775 tablet_service.cc:1467] Processing CreateTablet for tablet 0d555f059f84478eb805f4ca4534dab4 (DEFAULT_TABLE table=default.Table_99da18207fc542fbb68972724c4d410e [id=f75c1da3bd734e92bc105c473236fa9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:59.525919 11039 raft_consensus.cc:348] T 6ad838c884fc4b1d8f98781e7190e3df P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:59.526589 11039 raft_consensus.cc:374] T 6ad838c884fc4b1d8f98781e7190e3df P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:59.526916 11039 raft_consensus.cc:729] T 6ad838c884fc4b1d8f98781e7190e3df P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:20:59.527127 10775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d555f059f84478eb805f4ca4534dab4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:59.527591 11039 consensus_queue.cc:260] T 6ad838c884fc4b1d8f98781e7190e3df P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:59.531582 10373 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet ccb26e0fbc1948bc9a00e9e26c4d1139 (table default.Table_01b5209b9d5c4115b0bf6aef48b51bd8 [id=7e9e663acedf4a1fae2f04cde97b7907]). Expected version 1 got 0
I20260924 00:20:59.533869 10369 master_service.cc:946] ReplaceTablet: received request to replace tablet d44932f8f08e4ea7acc05b8ec511f045 from {username='slave'} at 127.0.0.1:47022
I20260924 00:20:59.534590 11040 tablet_bootstrap.cc:492] T c83942a74816465fa7f3f2c3e18c98bb P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:20:59.534955 11040 ts_tablet_manager.cc:1397] T c83942a74816465fa7f3f2c3e18c98bb P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.181s user 0.007s sys 0.002s
I20260924 00:20:59.539095 10634 tablet_service.cc:1467] Processing CreateTablet for tablet 119d83562ef546cab496cd7b44fecc15 (DEFAULT_TABLE table=default.Table_a6631f3ce9a04312b19aa2e85b4c9fe3 [id=5fb5e1e90c9440f385e7b04520bcba9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:59.540127 10634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 119d83562ef546cab496cd7b44fecc15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:59.540903 11039 ts_tablet_manager.cc:1428] T 6ad838c884fc4b1d8f98781e7190e3df P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.050s user 0.006s sys 0.001s
W20260924 00:20:59.542466 10371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9f2df24361da4f9a9340f6f8673043ea in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:59.549317 11039 tablet_bootstrap.cc:492] T ac559183fbef4e21b190a9a61a50e1e1 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:20:59.553856 11039 tablet_bootstrap.cc:654] T ac559183fbef4e21b190a9a61a50e1e1 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:59.559115 10916 tablet_service.cc:1467] Processing CreateTablet for tablet 119d83562ef546cab496cd7b44fecc15 (DEFAULT_TABLE table=default.Table_a6631f3ce9a04312b19aa2e85b4c9fe3 [id=5fb5e1e90c9440f385e7b04520bcba9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:59.560150 10916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 119d83562ef546cab496cd7b44fecc15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:59.560110 10773 tablet_service.cc:1467] Processing CreateTablet for tablet 2964cd68d5e94f3780a6a3157bd86983 (DEFAULT_TABLE table=default.Table_89e5d14641754be1934c7e3c880b21ee [id=fd5c26e6faab4fffb49cea70d6ab78a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:59.561167 10773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2964cd68d5e94f3780a6a3157bd86983. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:59.569460 10365 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_9608bad42525495eaa4df537b37969e7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:20:59.571187 10365 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9608bad42525495eaa4df537b37969e7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:59.574538 11041 tablet_bootstrap.cc:492] T c212525586544d8f9d92bc4b8bd8acaa P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:20:59.574939 11041 ts_tablet_manager.cc:1397] T c212525586544d8f9d92bc4b8bd8acaa P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.130s user 0.007s sys 0.001s
I20260924 00:20:59.577225 10919 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ad838c884fc4b1d8f98781e7190e3df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:57 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:20:59.577037 11040 raft_consensus.cc:348] T c83942a74816465fa7f3f2c3e18c98bb P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:59.577682 11040 raft_consensus.cc:374] T c83942a74816465fa7f3f2c3e18c98bb P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:59.577955 11040 raft_consensus.cc:729] T c83942a74816465fa7f3f2c3e18c98bb P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:20:59.578596 11040 consensus_queue.cc:260] T c83942a74816465fa7f3f2c3e18c98bb P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:59.580370 11040 ts_tablet_manager.cc:1428] T c83942a74816465fa7f3f2c3e18c98bb P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.045s user 0.001s sys 0.005s
I20260924 00:20:59.587682 11156 ts_tablet_manager.cc:1918] T 732a45a824274bdda8f7c060723a8cd2 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:59.588068 11156 log.cc:1196] T 732a45a824274bdda8f7c060723a8cd2 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/732a45a824274bdda8f7c060723a8cd2
I20260924 00:20:59.588667 11156 ts_tablet_manager.cc:1939] T 732a45a824274bdda8f7c060723a8cd2 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:20:59.589987 10919 tablet_service.cc:1467] Processing CreateTablet for tablet 2964cd68d5e94f3780a6a3157bd86983 (DEFAULT_TABLE table=default.Table_89e5d14641754be1934c7e3c880b21ee [id=fd5c26e6faab4fffb49cea70d6ab78a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:59.590967 10919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2964cd68d5e94f3780a6a3157bd86983. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:20:59.611649 11041 raft_consensus.cc:348] T c212525586544d8f9d92bc4b8bd8acaa P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:59.612540 11041 raft_consensus.cc:374] T c212525586544d8f9d92bc4b8bd8acaa P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:59.612608 10366 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet b06bf1e4691441769e162aa5404d4656
I20260924 00:20:59.612825 10374 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 2964cd68d5e94f3780a6a3157bd86983
I20260924 00:20:59.612799 11041 raft_consensus.cc:729] T c212525586544d8f9d92bc4b8bd8acaa P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:20:59.613385 11041 consensus_queue.cc:260] T c212525586544d8f9d92bc4b8bd8acaa P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:59.615572 11041 ts_tablet_manager.cc:1428] T c212525586544d8f9d92bc4b8bd8acaa P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.040s user 0.006s sys 0.000s
I20260924 00:20:59.615713 10915 tablet_service.cc:1514] Processing DeleteTablet for tablet b06bf1e4691441769e162aa5404d4656 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:59 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:20:59.616334 11041 tablet_bootstrap.cc:492] T ae0c9b785195491987dd8fe397d704e0 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:20:59.616436 10779 tablet_service.cc:1514] Processing DeleteTablet for tablet b06bf1e4691441769e162aa5404d4656 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:59 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:20:59.617075 10777 tablet_service.cc:1514] Processing DeleteTablet for tablet 2964cd68d5e94f3780a6a3157bd86983 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:59 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:20:59.617525 10920 tablet_service.cc:1514] Processing DeleteTablet for tablet 2964cd68d5e94f3780a6a3157bd86983 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:59 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:20:59.623067 11156 tablet_replica.cc:330] T b87bc7d0b65e4153aa649444fcc2c128 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:20:59.623530 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 732a45a824274bdda8f7c060723a8cd2 (table default.Table_ce92a0828938401292681552add70c45 [id=18a2080b0c7344a28df81315ee1e5cc3]) successfully deleted
I20260924 00:20:59.623725 11156 raft_consensus.cc:2229] T b87bc7d0b65e4153aa649444fcc2c128 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:59.624142 11156 raft_consensus.cc:2258] T b87bc7d0b65e4153aa649444fcc2c128 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:59.625272 11040 tablet_bootstrap.cc:492] T 1877e10d1db845f2a001063fb3bffef4 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:20:59.630553 10368 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet c83942a74816465fa7f3f2c3e18c98bb (table default.Table_41d1376556a9466fa348723fd8e2b2be [id=7eb84ebbe346450eb2b944db685c414b]). Expected version 3 got 0
I20260924 00:20:59.638059 10366 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_a54d4fd8353e42a4a00cb4e5c689da7b" } new_table_name: "default.Table_63927aa1872e41da917d6b92c8acb310" modify_external_catalogs: true
I20260924 00:20:59.640672 11039 tablet_bootstrap.cc:492] T ac559183fbef4e21b190a9a61a50e1e1 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:20:59.641068 11039 ts_tablet_manager.cc:1397] T ac559183fbef4e21b190a9a61a50e1e1 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.092s user 0.013s sys 0.002s
I20260924 00:20:59.643899 11051 ts_tablet_manager.cc:1918] T b87bc7d0b65e4153aa649444fcc2c128 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:20:59.646505 11051 log.cc:1196] T b87bc7d0b65e4153aa649444fcc2c128 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/b87bc7d0b65e4153aa649444fcc2c128
I20260924 00:20:59.644387 11039 raft_consensus.cc:348] T ac559183fbef4e21b190a9a61a50e1e1 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:59.647369 11039 raft_consensus.cc:374] T ac559183fbef4e21b190a9a61a50e1e1 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:59.646952 11150 raft_consensus.cc:482] T c83942a74816465fa7f3f2c3e18c98bb P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:20:59.647672 11039 raft_consensus.cc:729] T ac559183fbef4e21b190a9a61a50e1e1 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:20:59.647938 11150 raft_consensus.cc:504] T c83942a74816465fa7f3f2c3e18c98bb P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:59.648352 11039 consensus_queue.cc:260] T ac559183fbef4e21b190a9a61a50e1e1 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:59.650308 11039 ts_tablet_manager.cc:1428] T ac559183fbef4e21b190a9a61a50e1e1 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20260924 00:20:59.651114 11051 ts_tablet_manager.cc:1939] T b87bc7d0b65e4153aa649444fcc2c128 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:20:59.660202 11150 leader_election.cc:288] T c83942a74816465fa7f3f2c3e18c98bb P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:20:59.651901 11039 tablet_bootstrap.cc:492] T ccb26e0fbc1948bc9a00e9e26c4d1139 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:20:59.665146 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c83942a74816465fa7f3f2c3e18c98bb" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:20:59.665877 10942 raft_consensus.cc:2379] T c83942a74816465fa7f3f2c3e18c98bb P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48fbd603abe146c585d15196e1b5be08 in current term 1: Already voted for candidate c8626e300d0644b0908797ad61bd7cef in this term.
I20260924 00:20:59.669797 11040 tablet_bootstrap.cc:654] T 1877e10d1db845f2a001063fb3bffef4 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:59.672281 10370 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet c83942a74816465fa7f3f2c3e18c98bb (table default.Table_41d1376556a9466fa348723fd8e2b2be [id=7eb84ebbe346450eb2b944db685c414b]). Expected version 3 got 0
I20260924 00:20:59.673722 11041 tablet_bootstrap.cc:654] T ae0c9b785195491987dd8fe397d704e0 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:59.680927 10918 tablet_service.cc:1514] Processing DeleteTablet for tablet ac559183fbef4e21b190a9a61a50e1e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:56 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:20:59.682467 11049 ts_tablet_manager.cc:1918] T 453ce17089ce44ba8da67a0893301c3c P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:59.682560 11156 ts_tablet_manager.cc:1905] T b87bc7d0b65e4153aa649444fcc2c128 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:59.682822 11049 log.cc:1196] T 453ce17089ce44ba8da67a0893301c3c P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/453ce17089ce44ba8da67a0893301c3c
I20260924 00:20:59.683419 11049 ts_tablet_manager.cc:1939] T 453ce17089ce44ba8da67a0893301c3c P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:20:59.683177 10368 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_9abf361701cd4e6faee21d3e41dfeae4" } new_table_name: "default.Table_da80d8b26b9b48c79a00dd08f123f4f4" modify_external_catalogs: true
I20260924 00:20:59.692324 11039 tablet_bootstrap.cc:654] T ccb26e0fbc1948bc9a00e9e26c4d1139 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:59.695990 11051 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35358 (request call id 60) took 1864 ms. Trace:
I20260924 00:20:59.699733 11051 rpcz_store.cc:276] 0924 00:20:57.831921 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:20:57.832072 (+ 151us) service_pool.cc:227] Handling call
0924 00:20:59.274600 (+1442528us) ts_tablet_manager.cc:1145] Deleting tablet b87bc7d0b65e4153aa649444fcc2c128
0924 00:20:59.274646 (+ 46us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:20:59.578196 (+303550us) tablet_metadata.cc:641] Metadata flushed
0924 00:20:59.643683 (+ 65487us) tablet_metadata.cc:641] Metadata flushed
0924 00:20:59.695863 (+ 52180us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1441819}
I20260924 00:20:59.701103 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet b87bc7d0b65e4153aa649444fcc2c128 (table default.Table_dd9f87be029b4087b002975c2067d2f3 [id=87d4d63333db40a389607571eb2171ea]) successfully deleted
I20260924 00:20:59.702227 11051 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35358 (request call id 63) took 1632 ms. Trace:
I20260924 00:20:59.702562 11051 rpcz_store.cc:276] 0924 00:20:58.069777 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:20:58.069904 (+ 127us) service_pool.cc:227] Handling call
0924 00:20:59.701898 (+1631994us) ts_tablet_manager.cc:1145] Deleting tablet c212525586544d8f9d92bc4b8bd8acaa
0924 00:20:59.701938 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:20:59.702177 (+ 239us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1631401}
W20260924 00:20:59.703728 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet c212525586544d8f9d92bc4b8bd8acaa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c212525586544d8f9d92bc4b8bd8acaa already in progress: creating tablet
I20260924 00:20:59.709151 11041 tablet_bootstrap.cc:492] T ae0c9b785195491987dd8fe397d704e0 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:20:59.704689 11051 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35358 (request call id 64) took 1634 ms. Trace:
I20260924 00:20:59.710937 11151 raft_consensus.cc:482] T 1877e10d1db845f2a001063fb3bffef4 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:20:59.709564 11041 ts_tablet_manager.cc:1397] T ae0c9b785195491987dd8fe397d704e0 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.093s user 0.009s sys 0.002s
I20260924 00:20:59.712275 11151 raft_consensus.cc:504] T 1877e10d1db845f2a001063fb3bffef4 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:59.714171 11151 leader_election.cc:288] T 1877e10d1db845f2a001063fb3bffef4 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:20:59.716136 10799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1877e10d1db845f2a001063fb3bffef4" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:20:59.717561 10878 leader_election.cc:341] T 1877e10d1db845f2a001063fb3bffef4 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:20:59.718112 11051 rpcz_store.cc:276] 0924 00:20:58.070062 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:20:58.070460 (+ 398us) service_pool.cc:227] Handling call
0924 00:20:59.704407 (+1633947us) ts_tablet_manager.cc:1145] Deleting tablet 5d7d52be4f1f4fc29b8172b9fa878f76
0924 00:20:59.704442 (+ 35us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:20:59.704641 (+ 199us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1633358}
W20260924 00:20:59.720577 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 5d7d52be4f1f4fc29b8172b9fa878f76 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5d7d52be4f1f4fc29b8172b9fa878f76 already in progress: creating tablet
I20260924 00:20:59.721647 11051 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35358 (request call id 62) took 1652 ms. Trace:
I20260924 00:20:59.723412 11040 tablet_bootstrap.cc:492] T 1877e10d1db845f2a001063fb3bffef4 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:20:59.723827 11040 ts_tablet_manager.cc:1397] T 1877e10d1db845f2a001063fb3bffef4 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.099s user 0.012s sys 0.002s
I20260924 00:20:59.726917 11051 rpcz_store.cc:276] 0924 00:20:58.069447 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:20:58.213528 (+144081us) service_pool.cc:227] Handling call
0924 00:20:59.721335 (+1507807us) ts_tablet_manager.cc:1145] Deleting tablet 453ce17089ce44ba8da67a0893301c3c
0924 00:20:59.721377 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:20:59.721597 (+ 220us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1507002}
I20260924 00:20:59.730098 11051 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35358 (request call id 70) took 1065 ms. Trace:
I20260924 00:20:59.730310 11051 rpcz_store.cc:276] 0924 00:20:58.664669 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:20:58.829038 (+164369us) service_pool.cc:227] Handling call
0924 00:20:59.729888 (+900850us) ts_tablet_manager.cc:1145] Deleting tablet 86374d72238d42c29542e649a7a0200d
0924 00:20:59.729917 (+ 29us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:20:59.730062 (+ 145us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":900242}
I20260924 00:20:59.731129 11051 tablet_replica.cc:330] T 732a45a824274bdda8f7c060723a8cd2 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
W20260924 00:20:59.731086 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 453ce17089ce44ba8da67a0893301c3c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 453ce17089ce44ba8da67a0893301c3c already in progress: creating tablet
I20260924 00:20:59.731721 11051 raft_consensus.cc:2229] T 732a45a824274bdda8f7c060723a8cd2 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:59.732095 11051 raft_consensus.cc:2258] T 732a45a824274bdda8f7c060723a8cd2 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 00:20:59.732563 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 86374d72238d42c29542e649a7a0200d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 86374d72238d42c29542e649a7a0200d already in progress: creating tablet
I20260924 00:20:59.733633 11040 raft_consensus.cc:348] T 1877e10d1db845f2a001063fb3bffef4 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:59.734321 11040 raft_consensus.cc:374] T 1877e10d1db845f2a001063fb3bffef4 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:59.734673 11040 raft_consensus.cc:729] T 1877e10d1db845f2a001063fb3bffef4 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:20:59.735354 11040 consensus_queue.cc:260] T 1877e10d1db845f2a001063fb3bffef4 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:20:59.743522 11039 tablet_bootstrap.cc:492] T ccb26e0fbc1948bc9a00e9e26c4d1139 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:20:59.743898 11039 ts_tablet_manager.cc:1397] T ccb26e0fbc1948bc9a00e9e26c4d1139 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.092s user 0.009s sys 0.000s
I20260924 00:20:59.743999 11040 ts_tablet_manager.cc:1428] T 1877e10d1db845f2a001063fb3bffef4 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.016s user 0.005s sys 0.000s
I20260924 00:20:59.746616 11040 tablet_bootstrap.cc:492] T 6ad838c884fc4b1d8f98781e7190e3df P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:20:59.746450 11039 raft_consensus.cc:348] T ccb26e0fbc1948bc9a00e9e26c4d1139 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:59.747066 11039 raft_consensus.cc:374] T ccb26e0fbc1948bc9a00e9e26c4d1139 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:59.747298 11039 raft_consensus.cc:729] T ccb26e0fbc1948bc9a00e9e26c4d1139 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:20:59.749019 11051 ts_tablet_manager.cc:1905] T 732a45a824274bdda8f7c060723a8cd2 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:59.751648 11039 consensus_queue.cc:260] T ccb26e0fbc1948bc9a00e9e26c4d1139 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:59.756212 11040 tablet_bootstrap.cc:654] T 6ad838c884fc4b1d8f98781e7190e3df P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
W20260924 00:20:59.760610 10366 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e0074b1184ec4883ab1016330d98c968 consensus state has no leader
I20260924 00:20:59.775345 11039 ts_tablet_manager.cc:1428] T ccb26e0fbc1948bc9a00e9e26c4d1139 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.031s user 0.001s sys 0.005s
I20260924 00:20:59.778595 11039 tablet_bootstrap.cc:492] T 453ce17089ce44ba8da67a0893301c3c P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:20:59.779321 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 453ce17089ce44ba8da67a0893301c3c (table default.Table_ca083375f0df4255b8654ed88040dfcd [id=f79ca8d11aa04d5f8f72d58b798ed4ea]) successfully deleted
I20260924 00:20:59.792778 10633 tablet_service.cc:1467] Processing CreateTablet for tablet 2964cd68d5e94f3780a6a3157bd86983 (DEFAULT_TABLE table=default.Table_89e5d14641754be1934c7e3c880b21ee [id=fd5c26e6faab4fffb49cea70d6ab78a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:20:59.794890 10633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2964cd68d5e94f3780a6a3157bd86983. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:20:59.795701 10631 consensus_meta.cc:287] T 0980afc2af174da2a8ca2dd414e9825c P b10a9082a37b46f09d581944914c1670: Time spent flushing consensus metadata: real 0.532s user 0.007s sys 0.000s
W20260924 00:20:59.800544 10368 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 96960f3061fb43e9939b470f49edf1e9 consensus state has no leader
I20260924 00:20:59.801626 10634 tablet_service.cc:1514] Processing DeleteTablet for tablet 2964cd68d5e94f3780a6a3157bd86983 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:59 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:59.801970 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c83942a74816465fa7f3f2c3e18c98bb" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:20:59.802754 10635 tablet_service.cc:1514] Processing DeleteTablet for tablet b06bf1e4691441769e162aa5404d4656 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:59 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:59.803296 10654 raft_consensus.cc:2379] T c83942a74816465fa7f3f2c3e18c98bb P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48fbd603abe146c585d15196e1b5be08 in current term 1: Already voted for candidate b10a9082a37b46f09d581944914c1670 in this term.
I20260924 00:20:59.803573 10636 tablet_service.cc:1514] Processing DeleteTablet for tablet af2f58e6757a4b738a8ec38b2e228f9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:57 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:59.805159 10733 leader_election.cc:302] T c83942a74816465fa7f3f2c3e18c98bb P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08; no voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef
I20260924 00:20:59.806206 11150 raft_consensus.cc:3037] T c83942a74816465fa7f3f2c3e18c98bb P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:20:59.812497 11150 raft_consensus.cc:2726] T c83942a74816465fa7f3f2c3e18c98bb P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260924 00:20:59.813369 10876 leader_election.cc:341] T 1877e10d1db845f2a001063fb3bffef4 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: 1877e10d1db845f2a001063fb3bffef4
I20260924 00:20:59.813769 10876 leader_election.cc:302] T 1877e10d1db845f2a001063fb3bffef4 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670
I20260924 00:20:59.814738 11151 raft_consensus.cc:2726] T 1877e10d1db845f2a001063fb3bffef4 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:20:59.817129 10368 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_b22d604fe0dc4cad9d63baa9ffa5005c" } modify_external_catalogs: true
I20260924 00:20:59.817662 10368 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_b22d604fe0dc4cad9d63baa9ffa5005c" } modify_external_catalogs: true
I20260924 00:20:59.819401 11146 raft_consensus.cc:482] T e0074b1184ec4883ab1016330d98c968 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:20:59.820076 11146 raft_consensus.cc:504] T e0074b1184ec4883ab1016330d98c968 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:59.820627 11041 raft_consensus.cc:348] T ae0c9b785195491987dd8fe397d704e0 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:20:59.821269 11041 raft_consensus.cc:374] T ae0c9b785195491987dd8fe397d704e0 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:59.821550 11041 raft_consensus.cc:729] T ae0c9b785195491987dd8fe397d704e0 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:20:59.822508 11146 leader_election.cc:288] T e0074b1184ec4883ab1016330d98c968 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:20:59.822085 11041 consensus_queue.cc:260] T ae0c9b785195491987dd8fe397d704e0 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:20:59.823154 11049 tablet_replica.cc:330] T af2f58e6757a4b738a8ec38b2e228f9f P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:20:59.824898 10371 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_593b4d43f8144c04ab048e8221b3fc8e" } modify_external_catalogs: true
I20260924 00:20:59.825443 10371 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_593b4d43f8144c04ab048e8221b3fc8e" } modify_external_catalogs: true
I20260924 00:20:59.826327 10365 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_ec06ce66fea04103be3c890fc13493d9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:20:59.827898 11041 ts_tablet_manager.cc:1428] T ae0c9b785195491987dd8fe397d704e0 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.112s user 0.007s sys 0.000s
W20260924 00:20:59.827926 10365 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ec06ce66fea04103be3c890fc13493d9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:59.828903 11041 tablet_bootstrap.cc:492] T bcd8a6b92b8e492d96994195fa7f3ae5 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:20:59.829385 10799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0074b1184ec4883ab1016330d98c968" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:20:59.830935 10592 leader_election.cc:341] T e0074b1184ec4883ab1016330d98c968 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
W20260924 00:20:59.835076 10359 catalog_manager.cc:4808] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 2964cd68d5e94f3780a6a3157bd86983 because the tablet was not found. No further retry: Not found: Tablet not found: 2964cd68d5e94f3780a6a3157bd86983
W20260924 00:20:59.835709 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet b06bf1e4691441769e162aa5404d4656 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b06bf1e4691441769e162aa5404d4656 already in progress: creating tablet
I20260924 00:20:59.837209 11049 raft_consensus.cc:2229] T af2f58e6757a4b738a8ec38b2e228f9f P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:20:59.763803 11156 ts_tablet_manager.cc:1918] T b87bc7d0b65e4153aa649444fcc2c128 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:59.766553 10780 tablet_service.cc:1514] Processing DeleteTablet for tablet 1877e10d1db845f2a001063fb3bffef4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:56 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:20:59.839215 11156 log.cc:1196] T b87bc7d0b65e4153aa649444fcc2c128 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/b87bc7d0b65e4153aa649444fcc2c128
I20260924 00:20:59.839893 11156 ts_tablet_manager.cc:1939] T b87bc7d0b65e4153aa649444fcc2c128 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:20:59.842384 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet b87bc7d0b65e4153aa649444fcc2c128 (table default.Table_dd9f87be029b4087b002975c2067d2f3 [id=87d4d63333db40a389607571eb2171ea]) successfully deleted
W20260924 00:20:59.843087 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet d44932f8f08e4ea7acc05b8ec511f045 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d44932f8f08e4ea7acc05b8ec511f045 already in progress: creating tablet
W20260924 00:20:59.843721 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet eeb11092b803467e9db732468350e852 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet eeb11092b803467e9db732468350e852 already in progress: creating tablet
I20260924 00:20:59.843942 11156 tablet_replica.cc:330] T 1877e10d1db845f2a001063fb3bffef4 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
W20260924 00:20:59.844326 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet b06bf1e4691441769e162aa5404d4656 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b06bf1e4691441769e162aa5404d4656 already in progress: creating tablet
I20260924 00:20:59.844628 11156 raft_consensus.cc:2229] T 1877e10d1db845f2a001063fb3bffef4 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260924 00:20:59.845506 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 2964cd68d5e94f3780a6a3157bd86983 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2964cd68d5e94f3780a6a3157bd86983 already in progress: creating tablet
I20260924 00:20:59.847818 11039 tablet_bootstrap.cc:654] T 453ce17089ce44ba8da67a0893301c3c P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:59.860286 11040 tablet_bootstrap.cc:492] T 6ad838c884fc4b1d8f98781e7190e3df P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:20:59.861639 11040 ts_tablet_manager.cc:1397] T 6ad838c884fc4b1d8f98781e7190e3df P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.117s user 0.006s sys 0.005s
I20260924 00:20:59.865705 10636 tablet_service.cc:1514] Processing DeleteTablet for tablet c212525586544d8f9d92bc4b8bd8acaa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:57 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:20:59.865844 11049 raft_consensus.cc:2258] T af2f58e6757a4b738a8ec38b2e228f9f P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:59.869288 10365 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_fe2e015f4c3d44f5a3ed8797af219464"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:20:59.869985 11040 raft_consensus.cc:348] T 6ad838c884fc4b1d8f98781e7190e3df P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:59.870643 11040 raft_consensus.cc:374] T 6ad838c884fc4b1d8f98781e7190e3df P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:59.870882 11040 raft_consensus.cc:729] T 6ad838c884fc4b1d8f98781e7190e3df P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:20:59.873409 10370 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_236b1998bb2f474c9e5452abe4c83c64"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:20:59.881260 10374 master_service.cc:946] ReplaceTablet: received request to replace tablet c83942a74816465fa7f3f2c3e18c98bb from {username='slave'} at 127.0.0.1:47022
I20260924 00:20:59.883242 11041 tablet_bootstrap.cc:654] T bcd8a6b92b8e492d96994195fa7f3ae5 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:20:59.883527 11049 ts_tablet_manager.cc:1905] T af2f58e6757a4b738a8ec38b2e228f9f P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260924 00:20:59.885042 10370 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_236b1998bb2f474c9e5452abe4c83c64 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260924 00:20:59.885072 10365 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fe2e015f4c3d44f5a3ed8797af219464 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:59.902125 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0074b1184ec4883ab1016330d98c968" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
W20260924 00:20:59.907091 10589 leader_election.cc:341] T e0074b1184ec4883ab1016330d98c968 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:20:59.908124 10370 master_service.cc:946] ReplaceTablet: received request to replace tablet ffa011cf8dc446de849bea75a4f20056 from {username='slave'} at 127.0.0.1:47022
I20260924 00:20:59.908638 10589 leader_election.cc:302] T e0074b1184ec4883ab1016330d98c968 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:20:59.909435 11156 raft_consensus.cc:2258] T 1877e10d1db845f2a001063fb3bffef4 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:20:59.910503 10369 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_08bc69c89b564e158a0b6d764972cb19"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:20:59.912137 11156 ts_tablet_manager.cc:1905] T 1877e10d1db845f2a001063fb3bffef4 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:20:59.916267 11191 raft_consensus.cc:2726] T e0074b1184ec4883ab1016330d98c968 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260924 00:20:59.925770 10369 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_08bc69c89b564e158a0b6d764972cb19 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:20:59.925547 11041 tablet_bootstrap.cc:492] T bcd8a6b92b8e492d96994195fa7f3ae5 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:20:59.929294 11041 ts_tablet_manager.cc:1397] T bcd8a6b92b8e492d96994195fa7f3ae5 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.101s user 0.011s sys 0.002s
I20260924 00:20:59.930058 11051 ts_tablet_manager.cc:1918] T 732a45a824274bdda8f7c060723a8cd2 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:20:59.933269 11041 raft_consensus.cc:348] T bcd8a6b92b8e492d96994195fa7f3ae5 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:59.936558 11040 consensus_queue.cc:260] T 6ad838c884fc4b1d8f98781e7190e3df P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:59.937373 11041 raft_consensus.cc:374] T bcd8a6b92b8e492d96994195fa7f3ae5 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:20:59.937693 11041 raft_consensus.cc:729] T bcd8a6b92b8e492d96994195fa7f3ae5 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:20:59.938850 11150 raft_consensus.cc:482] T 6ad838c884fc4b1d8f98781e7190e3df P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:20:59.939221 11150 raft_consensus.cc:504] T 6ad838c884fc4b1d8f98781e7190e3df P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:59.943778 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ad838c884fc4b1d8f98781e7190e3df" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:20:59.944377 10942 raft_consensus.cc:2454] T 6ad838c884fc4b1d8f98781e7190e3df P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 0.
I20260924 00:20:59.945757 11150 leader_election.cc:288] T 6ad838c884fc4b1d8f98781e7190e3df P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
W20260924 00:20:59.946568 10733 leader_election.cc:341] T 6ad838c884fc4b1d8f98781e7190e3df P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: 6ad838c884fc4b1d8f98781e7190e3df
I20260924 00:20:59.947449 10732 leader_election.cc:302] T 6ad838c884fc4b1d8f98781e7190e3df P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters: b10a9082a37b46f09d581944914c1670
I20260924 00:20:59.949517 11150 raft_consensus.cc:2781] T 6ad838c884fc4b1d8f98781e7190e3df P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:20:59.949868 11150 raft_consensus.cc:482] T 6ad838c884fc4b1d8f98781e7190e3df P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:20:59.950197 11150 raft_consensus.cc:3037] T 6ad838c884fc4b1d8f98781e7190e3df P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:20:59.956585 11150 raft_consensus.cc:504] T 6ad838c884fc4b1d8f98781e7190e3df P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:59.930428 11051 log.cc:1196] T 732a45a824274bdda8f7c060723a8cd2 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/732a45a824274bdda8f7c060723a8cd2
I20260924 00:20:59.958603 11051 ts_tablet_manager.cc:1939] T 732a45a824274bdda8f7c060723a8cd2 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:20:59.959559 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ad838c884fc4b1d8f98781e7190e3df" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef"
I20260924 00:20:59.960071 10942 raft_consensus.cc:3037] T 6ad838c884fc4b1d8f98781e7190e3df P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:20:59.960387 11051 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35358 (request call id 72) took 1070 ms. Trace:
I20260924 00:20:59.964808 10369 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_fcb196dda8a044e8b443fa810c5b8d41" } new_table_name: "default.Table_1347839140d544c3b4aee063e2efe906" modify_external_catalogs: true
I20260924 00:20:59.967639 10942 raft_consensus.cc:2454] T 6ad838c884fc4b1d8f98781e7190e3df P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 1.
I20260924 00:20:59.968973 10732 leader_election.cc:302] T 6ad838c884fc4b1d8f98781e7190e3df P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:20:59.938400 11041 consensus_queue.cc:260] T bcd8a6b92b8e492d96994195fa7f3ae5 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:20:59.960652 11051 rpcz_store.cc:276] 0924 00:20:58.889493 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:20:58.892020 (+ 2527us) service_pool.cc:227] Handling call
0924 00:20:59.731003 (+838983us) ts_tablet_manager.cc:1145] Deleting tablet 732a45a824274bdda8f7c060723a8cd2
0924 00:20:59.731034 (+ 31us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:20:59.787810 (+ 56776us) tablet_metadata.cc:641] Metadata flushed
0924 00:20:59.929835 (+142025us) tablet_metadata.cc:641] Metadata flushed
0924 00:20:59.960333 (+ 30498us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":838450}
W20260924 00:20:59.983384 10369 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 666755ec87bb496db75e43ec9204ec5b consensus state has no leader
I20260924 00:20:59.988967 11041 ts_tablet_manager.cc:1428] T bcd8a6b92b8e492d96994195fa7f3ae5 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.059s user 0.005s sys 0.000s
I20260924 00:20:59.989876 11041 tablet_bootstrap.cc:492] T eeb11092b803467e9db732468350e852 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:20:59.984154 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 732a45a824274bdda8f7c060723a8cd2 (table default.Table_ce92a0828938401292681552add70c45 [id=18a2080b0c7344a28df81315ee1e5cc3]) successfully deleted
I20260924 00:20:59.997174 11039 tablet_bootstrap.cc:492] T 453ce17089ce44ba8da67a0893301c3c P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
W20260924 00:20:59.999948 10733 leader_election.cc:341] T 6ad838c884fc4b1d8f98781e7190e3df P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: 6ad838c884fc4b1d8f98781e7190e3df
I20260924 00:21:00.000509 11150 leader_election.cc:288] T 6ad838c884fc4b1d8f98781e7190e3df P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Requested vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:20:59.993058 11197 raft_consensus.cc:2781] T 6ad838c884fc4b1d8f98781e7190e3df P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:00.001251 11197 raft_consensus.cc:686] T 6ad838c884fc4b1d8f98781e7190e3df P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: Becoming Leader. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Running, Role: LEADER
W20260924 00:21:00.001657 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet d44932f8f08e4ea7acc05b8ec511f045 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d44932f8f08e4ea7acc05b8ec511f045 already in progress: creating tablet
I20260924 00:21:00.001996 11197 consensus_queue.cc:237] T 6ad838c884fc4b1d8f98781e7190e3df P 48fbd603abe146c585d15196e1b5be08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
W20260924 00:21:00.005843 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet eeb11092b803467e9db732468350e852 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet eeb11092b803467e9db732468350e852 already in progress: creating tablet
I20260924 00:21:00.007256 10369 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_f1bc26fec8e746ba84ca80afe98a4d26" } new_table_name: "default.Table_a5a83435a7fb47ce8083022c0686b74a" modify_external_catalogs: true
I20260924 00:21:00.007536 11040 ts_tablet_manager.cc:1428] T 6ad838c884fc4b1d8f98781e7190e3df P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.145s user 0.005s sys 0.000s
I20260924 00:21:00.010181 11040 tablet_bootstrap.cc:492] T ac559183fbef4e21b190a9a61a50e1e1 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:00.012768 11049 ts_tablet_manager.cc:1918] T af2f58e6757a4b738a8ec38b2e228f9f P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:00.013224 11049 log.cc:1196] T af2f58e6757a4b738a8ec38b2e228f9f P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/af2f58e6757a4b738a8ec38b2e228f9f
I20260924 00:21:00.014046 11049 ts_tablet_manager.cc:1939] T af2f58e6757a4b738a8ec38b2e228f9f P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:00.014961 11040 tablet_bootstrap.cc:654] T ac559183fbef4e21b190a9a61a50e1e1 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:00.016498 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet af2f58e6757a4b738a8ec38b2e228f9f (table default.Table_391ed8efc7a54515837c1b79be85d64c [id=f72ddb5979044cda93de75fcb1983ca4]) successfully deleted
I20260924 00:21:00.017421 11049 tablet_replica.cc:330] T c212525586544d8f9d92bc4b8bd8acaa P b10a9082a37b46f09d581944914c1670: stopping tablet replica
W20260924 00:21:00.018831 10732 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:41301, user_credentials={real_user=slave}} blocked reactor thread for 49979.2us
I20260924 00:21:00.020190 11039 ts_tablet_manager.cc:1397] T 453ce17089ce44ba8da67a0893301c3c P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.242s user 0.009s sys 0.006s
I20260924 00:21:00.019706 11051 tablet_replica.cc:330] T 1877e10d1db845f2a001063fb3bffef4 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:00.026283 11051 raft_consensus.cc:2229] T 1877e10d1db845f2a001063fb3bffef4 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:00.026716 11051 raft_consensus.cc:2258] T 1877e10d1db845f2a001063fb3bffef4 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:00.027293 11041 tablet_bootstrap.cc:654] T eeb11092b803467e9db732468350e852 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:00.030076 11156 ts_tablet_manager.cc:1918] T 1877e10d1db845f2a001063fb3bffef4 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:00.030517 11156 log.cc:1196] T 1877e10d1db845f2a001063fb3bffef4 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/1877e10d1db845f2a001063fb3bffef4
W20260924 00:21:00.031116 10369 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 18abdec265a24dc8bfc765d4395767c0 has no consensus state
I20260924 00:21:00.031239 11156 ts_tablet_manager.cc:1939] T 1877e10d1db845f2a001063fb3bffef4 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:00.034263 11049 raft_consensus.cc:2229] T c212525586544d8f9d92bc4b8bd8acaa P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:00.034739 11049 raft_consensus.cc:2258] T c212525586544d8f9d92bc4b8bd8acaa P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:00.040581 11040 tablet_bootstrap.cc:492] T ac559183fbef4e21b190a9a61a50e1e1 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:00.042106 11049 ts_tablet_manager.cc:1905] T c212525586544d8f9d92bc4b8bd8acaa P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:00.046281 10780 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ad838c884fc4b1d8f98781e7190e3df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:57 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:00.055171 10780 tablet_service.cc:1467] Processing CreateTablet for tablet f3432ac96da94d5099bcf9764c2b32e1 (DEFAULT_TABLE table=default.Table_9608bad42525495eaa4df537b37969e7 [id=aa74d52589984aec8e6d296e5dffd623]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:00.056020 11040 ts_tablet_manager.cc:1397] T ac559183fbef4e21b190a9a61a50e1e1 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.033s user 0.010s sys 0.000s
I20260924 00:21:00.055918 10366 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet ccb26e0fbc1948bc9a00e9e26c4d1139 (table default.Table_01b5209b9d5c4115b0bf6aef48b51bd8 [id=7e9e663acedf4a1fae2f04cde97b7907]). Expected version 1 got 0
I20260924 00:21:00.056125 10780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3432ac96da94d5099bcf9764c2b32e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:00.059954 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 1877e10d1db845f2a001063fb3bffef4 (table default.Table_5f8c295237ab4e0ea184a03a9fa458a2 [id=be698d8b82734ff5940f6915ba1d4df1]) successfully deleted
I20260924 00:21:00.061112 11156 tablet_replica.cc:330] T 6ad838c884fc4b1d8f98781e7190e3df P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:00.061902 11156 raft_consensus.cc:2229] T 6ad838c884fc4b1d8f98781e7190e3df P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:21:00.063539 11156 pending_rounds.cc:62] T 6ad838c884fc4b1d8f98781e7190e3df P 48fbd603abe146c585d15196e1b5be08: Trying to abort 1 pending ops.
I20260924 00:21:00.064461 11156 pending_rounds.cc:68] T 6ad838c884fc4b1d8f98781e7190e3df P 48fbd603abe146c585d15196e1b5be08: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332697129055436800 op_type: NO_OP noop_request { }
I20260924 00:21:00.065392 11156 raft_consensus.cc:2866] T 6ad838c884fc4b1d8f98781e7190e3df P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260924 00:21:00.066188 11156 raft_consensus.cc:2258] T 6ad838c884fc4b1d8f98781e7190e3df P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:00.060637 10367 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet ae0c9b785195491987dd8fe397d704e0 (table default.Table_d61af65b53ce4871b7b24752281af745 [id=0edc6c93a1ba4c8d81755129301c83de]). Expected version 1 got 0
I20260924 00:21:00.058482 11040 raft_consensus.cc:348] T ac559183fbef4e21b190a9a61a50e1e1 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:00.067207 11040 raft_consensus.cc:374] T ac559183fbef4e21b190a9a61a50e1e1 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:00.067440 11040 raft_consensus.cc:729] T ac559183fbef4e21b190a9a61a50e1e1 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:00.068073 11040 consensus_queue.cc:260] T ac559183fbef4e21b190a9a61a50e1e1 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:00.069257 10366 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_dae0ffc8534047588f6f7cb13a610af5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:00.074299 10775 tablet_service.cc:1467] Processing CreateTablet for tablet 5e624be739394e6d87d6b3fef2f46302 (DEFAULT_TABLE table=default.Table_e86eeb97d87d4f9ea7edfe269cc1d790 [id=df2a5a74db65431c9ecb750184f64709]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:00.075205 10775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e624be739394e6d87d6b3fef2f46302. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:21:00.073915 10366 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dae0ffc8534047588f6f7cb13a610af5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:00.078292 10774 tablet_service.cc:1467] Processing CreateTablet for tablet 5f2a32d5afff458fab79ffa106a98884 (DEFAULT_TABLE table=default.Table_ff66b5b591e74da99ad87f8512cdc082 [id=4f70e833d7f74d91af0faa01ed52f78b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:00.079416 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f2a32d5afff458fab79ffa106a98884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:00.084442 10369 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet ccb26e0fbc1948bc9a00e9e26c4d1139 (table default.Table_01b5209b9d5c4115b0bf6aef48b51bd8 [id=7e9e663acedf4a1fae2f04cde97b7907]). Expected version 1 got 0
I20260924 00:21:00.091595 10773 tablet_service.cc:1467] Processing CreateTablet for tablet bf144ab1a6464a4ab02ac0642c5255e3 (DEFAULT_TABLE table=default.Table_a7b8969040dd4ef5ad547c617f03d471 [id=3db896d304514ead8dfe726c402a6fcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:00.092677 10773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf144ab1a6464a4ab02ac0642c5255e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:00.094682 10776 tablet_service.cc:1514] Processing DeleteTablet for tablet ac559183fbef4e21b190a9a61a50e1e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:56 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:00.096850 10633 tablet_service.cc:1467] Processing CreateTablet for tablet f3432ac96da94d5099bcf9764c2b32e1 (DEFAULT_TABLE table=default.Table_9608bad42525495eaa4df537b37969e7 [id=aa74d52589984aec8e6d296e5dffd623]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:00.098028 10636 tablet_service.cc:1467] Processing CreateTablet for tablet 5e624be739394e6d87d6b3fef2f46302 (DEFAULT_TABLE table=default.Table_e86eeb97d87d4f9ea7edfe269cc1d790 [id=df2a5a74db65431c9ecb750184f64709]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:00.099355 10635 tablet_service.cc:1467] Processing CreateTablet for tablet 5f2a32d5afff458fab79ffa106a98884 (DEFAULT_TABLE table=default.Table_ff66b5b591e74da99ad87f8512cdc082 [id=4f70e833d7f74d91af0faa01ed52f78b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:00.109174 10634 tablet_service.cc:1514] Processing DeleteTablet for tablet bcd8a6b92b8e492d96994195fa7f3ae5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:58 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:00.110535 10374 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet c83942a74816465fa7f3f2c3e18c98bb
I20260924 00:21:00.111260 11041 tablet_bootstrap.cc:492] T eeb11092b803467e9db732468350e852 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:00.111626 11041 ts_tablet_manager.cc:1397] T eeb11092b803467e9db732468350e852 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.122s user 0.010s sys 0.001s
I20260924 00:21:00.111788 10374 catalog_manager.cc:6565] ReplaceTablet: tablet c83942a74816465fa7f3f2c3e18c98bb deleted and replaced by tablet 2861c52b3a1e4e36a45e7f4e5d65ac31
I20260924 00:21:00.113652 11041 raft_consensus.cc:348] T eeb11092b803467e9db732468350e852 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:00.114239 11041 raft_consensus.cc:374] T eeb11092b803467e9db732468350e852 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:00.114461 11041 raft_consensus.cc:729] T eeb11092b803467e9db732468350e852 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:00.114987 10634 tablet_service.cc:1514] Processing DeleteTablet for tablet c83942a74816465fa7f3f2c3e18c98bb with delete_type TABLET_DATA_DELETED (replaced by tablet 2861c52b3a1e4e36a45e7f4e5d65ac31) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:00.115906 11040 ts_tablet_manager.cc:1428] T ac559183fbef4e21b190a9a61a50e1e1 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.059s user 0.001s sys 0.003s
I20260924 00:21:00.116604 10776 tablet_service.cc:1514] Processing DeleteTablet for tablet c83942a74816465fa7f3f2c3e18c98bb with delete_type TABLET_DATA_DELETED (replaced by tablet 2861c52b3a1e4e36a45e7f4e5d65ac31) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:00.119884 10374 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_9f40e044dcc74a8d90f5fa80c9ec7a26"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:00.122609 10374 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9f40e044dcc74a8d90f5fa80c9ec7a26 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:00.121109 10371 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet c7e50ac8b7f947b1bfba8cd1cff0ea7e
I20260924 00:21:00.123238 11156 ts_tablet_manager.cc:1905] T 6ad838c884fc4b1d8f98781e7190e3df P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:00.063217 10918 tablet_service.cc:1467] Processing CreateTablet for tablet f3432ac96da94d5099bcf9764c2b32e1 (DEFAULT_TABLE table=default.Table_9608bad42525495eaa4df537b37969e7 [id=aa74d52589984aec8e6d296e5dffd623]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:00.124993 11051 ts_tablet_manager.cc:1905] T 1877e10d1db845f2a001063fb3bffef4 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:00.125100 10918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3432ac96da94d5099bcf9764c2b32e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:00.075032 10917 tablet_service.cc:1467] Processing CreateTablet for tablet 5e624be739394e6d87d6b3fef2f46302 (DEFAULT_TABLE table=default.Table_e86eeb97d87d4f9ea7edfe269cc1d790 [id=df2a5a74db65431c9ecb750184f64709]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:00.129590 10917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e624be739394e6d87d6b3fef2f46302. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:00.078094 10916 tablet_service.cc:1467] Processing CreateTablet for tablet 5f2a32d5afff458fab79ffa106a98884 (DEFAULT_TABLE table=default.Table_ff66b5b591e74da99ad87f8512cdc082 [id=4f70e833d7f74d91af0faa01ed52f78b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:00.133603 10916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f2a32d5afff458fab79ffa106a98884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:00.126858 11039 raft_consensus.cc:348] T 453ce17089ce44ba8da67a0893301c3c P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:00.133955 11039 raft_consensus.cc:374] T 453ce17089ce44ba8da67a0893301c3c P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:00.134203 11039 raft_consensus.cc:729] T 453ce17089ce44ba8da67a0893301c3c P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:00.136732 11041 consensus_queue.cc:260] T eeb11092b803467e9db732468350e852 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:00.118606 10919 tablet_service.cc:1514] Processing DeleteTablet for tablet c83942a74816465fa7f3f2c3e18c98bb with delete_type TABLET_DATA_DELETED (replaced by tablet 2861c52b3a1e4e36a45e7f4e5d65ac31) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:00.143935 10630 tablet_service.cc:1467] Processing CreateTablet for tablet 95a7a1d2147c4d7ea63ca37a8f0ea4e6 (DEFAULT_TABLE table=default.Table_95d86f2882084cb6821c2aa52497b554 [id=c0cd199f74494deda9d088345c4c90a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:00.145349 11041 ts_tablet_manager.cc:1428] T eeb11092b803467e9db732468350e852 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.033s user 0.006s sys 0.000s
I20260924 00:21:00.145804 10629 tablet_service.cc:1467] Processing CreateTablet for tablet 4325b60efc1649c9ba4828d0a098a676 (DEFAULT_TABLE table=default.Table_0e5a147af0824c1d888bee1c066cb368 [id=7a76983970cd491a8e8635db379bdd10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:00.143970 10631 tablet_service.cc:1467] Processing CreateTablet for tablet bf144ab1a6464a4ab02ac0642c5255e3 (DEFAULT_TABLE table=default.Table_a7b8969040dd4ef5ad547c617f03d471 [id=3db896d304514ead8dfe726c402a6fcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:00.144721 10632 tablet_service.cc:1467] Processing CreateTablet for tablet 4420cead1f284af8958f189557833098 (DEFAULT_TABLE table=default.Table_cca0dd1d3cd247398f21a23a2724f3fc [id=21e1b3346ba346c883fced1e7a1b6cf6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:00.148007 10634 tablet_service.cc:1514] Processing DeleteTablet for tablet c7e50ac8b7f947b1bfba8cd1cff0ea7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:59 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:00.148274 11040 tablet_bootstrap.cc:492] T 453ce17089ce44ba8da67a0893301c3c P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:00.143062 11039 consensus_queue.cc:260] T 453ce17089ce44ba8da67a0893301c3c P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:00.153118 10773 tablet_service.cc:1514] Processing DeleteTablet for tablet c7e50ac8b7f947b1bfba8cd1cff0ea7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:59 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:00.153153 10775 tablet_service.cc:1467] Processing CreateTablet for tablet 95a7a1d2147c4d7ea63ca37a8f0ea4e6 (DEFAULT_TABLE table=default.Table_95d86f2882084cb6821c2aa52497b554 [id=c0cd199f74494deda9d088345c4c90a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:00.154281 10775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95a7a1d2147c4d7ea63ca37a8f0ea4e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:00.154381 10776 tablet_service.cc:1467] Processing CreateTablet for tablet 4325b60efc1649c9ba4828d0a098a676 (DEFAULT_TABLE table=default.Table_0e5a147af0824c1d888bee1c066cb368 [id=7a76983970cd491a8e8635db379bdd10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:00.155439 10776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4325b60efc1649c9ba4828d0a098a676. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:00.157464 11146 raft_consensus.cc:482] T eeb11092b803467e9db732468350e852 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:00.157522 10780 tablet_service.cc:1467] Processing CreateTablet for tablet 4420cead1f284af8958f189557833098 (DEFAULT_TABLE table=default.Table_cca0dd1d3cd247398f21a23a2724f3fc [id=21e1b3346ba346c883fced1e7a1b6cf6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:00.157910 11146 raft_consensus.cc:504] T eeb11092b803467e9db732468350e852 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:00.159754 11146 leader_election.cc:288] T eeb11092b803467e9db732468350e852 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:00.162312 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eeb11092b803467e9db732468350e852" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:00.168262 10780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4420cead1f284af8958f189557833098. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:21:00.168644 10589 leader_election.cc:341] T eeb11092b803467e9db732468350e852 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:00.169256 10919 tablet_service.cc:1514] Processing DeleteTablet for tablet c7e50ac8b7f947b1bfba8cd1cff0ea7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:59 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:00.170429 10921 tablet_service.cc:1467] Processing CreateTablet for tablet bf144ab1a6464a4ab02ac0642c5255e3 (DEFAULT_TABLE table=default.Table_a7b8969040dd4ef5ad547c617f03d471 [id=3db896d304514ead8dfe726c402a6fcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:00.172108 11197 raft_consensus.cc:482] T ac559183fbef4e21b190a9a61a50e1e1 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:00.172490 11197 raft_consensus.cc:504] T ac559183fbef4e21b190a9a61a50e1e1 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:00.172765 10919 tablet_service.cc:1467] Processing CreateTablet for tablet 4420cead1f284af8958f189557833098 (DEFAULT_TABLE table=default.Table_cca0dd1d3cd247398f21a23a2724f3fc [id=21e1b3346ba346c883fced1e7a1b6cf6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:00.174165 10920 tablet_service.cc:1467] Processing CreateTablet for tablet 95a7a1d2147c4d7ea63ca37a8f0ea4e6 (DEFAULT_TABLE table=default.Table_95d86f2882084cb6821c2aa52497b554 [id=c0cd199f74494deda9d088345c4c90a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:00.174459 11197 leader_election.cc:288] T ac559183fbef4e21b190a9a61a50e1e1 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:00.176838 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac559183fbef4e21b190a9a61a50e1e1" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:00.177414 10942 raft_consensus.cc:2454] T ac559183fbef4e21b190a9a61a50e1e1 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 0.
I20260924 00:21:00.178457 10732 leader_election.cc:302] T ac559183fbef4e21b190a9a61a50e1e1 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:00.166249 11039 ts_tablet_manager.cc:1428] T 453ce17089ce44ba8da67a0893301c3c P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.146s user 0.005s sys 0.000s
W20260924 00:21:00.179703 10733 leader_election.cc:341] T ac559183fbef4e21b190a9a61a50e1e1 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: ac559183fbef4e21b190a9a61a50e1e1
I20260924 00:21:00.180734 11197 raft_consensus.cc:2781] T ac559183fbef4e21b190a9a61a50e1e1 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:00.181111 11197 raft_consensus.cc:482] T ac559183fbef4e21b190a9a61a50e1e1 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:00.181398 11197 raft_consensus.cc:3037] T ac559183fbef4e21b190a9a61a50e1e1 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:00.185760 10799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eeb11092b803467e9db732468350e852" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:00.186965 11039 tablet_bootstrap.cc:492] T e0074b1184ec4883ab1016330d98c968 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
W20260924 00:21:00.186975 10592 leader_election.cc:341] T eeb11092b803467e9db732468350e852 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:00.187342 10592 leader_election.cc:302] T eeb11092b803467e9db732468350e852 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:00.188409 11146 raft_consensus.cc:2726] T eeb11092b803467e9db732468350e852 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:00.192174 10373 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_cca0dd1d3cd247398f21a23a2724f3fc" } new_table_name: "default.Table_bdbcdcde9ac241d5b22b5e8b0da6a9a7" modify_external_catalogs: true
I20260924 00:21:00.192102 10365 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_1ebdc131b4ce4446b021aca0fd94ae99"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:00.193523 10915 tablet_service.cc:1514] Processing DeleteTablet for tablet 453ce17089ce44ba8da67a0893301c3c with delete_type TABLET_DATA_DELETED (replaced by tablet 195b033091b543b58ee08417eae2c0e6) from {username='slave'} at 127.0.0.1:35358
W20260924 00:21:00.193819 10365 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1ebdc131b4ce4446b021aca0fd94ae99 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:00.193898 10371 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_d56dcf7ebba04b769e2e233cc7d9fa0e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:00.195465 10371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d56dcf7ebba04b769e2e233cc7d9fa0e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:00.195894 11041 tablet_bootstrap.cc:492] T 1ec36d59b5114feea144bcb72794136c P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:00.202265 11041 tablet_bootstrap.cc:654] T 1ec36d59b5114feea144bcb72794136c P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:00.205011 11197 raft_consensus.cc:504] T ac559183fbef4e21b190a9a61a50e1e1 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:00.207963 10774 tablet_service.cc:1467] Processing CreateTablet for tablet 18abdec265a24dc8bfc765d4395767c0 (DEFAULT_TABLE table=default.Table_a5a83435a7fb47ce8083022c0686b74a [id=a587a50b0c204843a41acacc45c2b150]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:00.207945 10915 tablet_service.cc:1467] Processing CreateTablet for tablet 4325b60efc1649c9ba4828d0a098a676 (DEFAULT_TABLE table=default.Table_0e5a147af0824c1d888bee1c066cb368 [id=7a76983970cd491a8e8635db379bdd10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:00.208506 10922 tablet_service.cc:1467] Processing CreateTablet for tablet 18abdec265a24dc8bfc765d4395767c0 (DEFAULT_TABLE table=default.Table_a5a83435a7fb47ce8083022c0686b74a [id=a587a50b0c204843a41acacc45c2b150]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:00.209124 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18abdec265a24dc8bfc765d4395767c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:00.209221 10634 tablet_service.cc:1467] Processing CreateTablet for tablet 18abdec265a24dc8bfc765d4395767c0 (DEFAULT_TABLE table=default.Table_a5a83435a7fb47ce8083022c0686b74a [id=a587a50b0c204843a41acacc45c2b150]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:00.214109 10635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f2a32d5afff458fab79ffa106a98884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:00.214596 10633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3432ac96da94d5099bcf9764c2b32e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:00.215116 10634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18abdec265a24dc8bfc765d4395767c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:00.215965 10631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf144ab1a6464a4ab02ac0642c5255e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:00.216831 10632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4420cead1f284af8958f189557833098. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:00.153206 11040 tablet_bootstrap.cc:654] T 453ce17089ce44ba8da67a0893301c3c P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:00.220857 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac559183fbef4e21b190a9a61a50e1e1" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef"
I20260924 00:21:00.221212 10630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95a7a1d2147c4d7ea63ca37a8f0ea4e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:00.221452 10942 raft_consensus.cc:3037] T ac559183fbef4e21b190a9a61a50e1e1 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:00.222395 10629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4325b60efc1649c9ba4828d0a098a676. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:00.223721 10636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e624be739394e6d87d6b3fef2f46302. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:21:00.224897 10733 leader_election.cc:341] T ac559183fbef4e21b190a9a61a50e1e1 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: ac559183fbef4e21b190a9a61a50e1e1
I20260924 00:21:00.226300 11197 leader_election.cc:288] T ac559183fbef4e21b190a9a61a50e1e1 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Requested vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:00.238559 11040 tablet_bootstrap.cc:492] T 453ce17089ce44ba8da67a0893301c3c P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:00.238902 11040 ts_tablet_manager.cc:1397] T 453ce17089ce44ba8da67a0893301c3c P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.091s user 0.004s sys 0.004s
I20260924 00:21:00.240406 10369 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_bc2eec8db7494414acf50558e06cd11e" } new_table_name: "default.Table_de36c9f702024914965bc6b371eda68e" modify_external_catalogs: true
I20260924 00:21:00.241472 11040 raft_consensus.cc:348] T 453ce17089ce44ba8da67a0893301c3c P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:00.242163 10628 tablet_service.cc:1514] Processing DeleteTablet for tablet eeb11092b803467e9db732468350e852 with delete_type TABLET_DATA_DELETED (replaced by tablet 4325b60efc1649c9ba4828d0a098a676) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:00.247447 10942 raft_consensus.cc:2454] T ac559183fbef4e21b190a9a61a50e1e1 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 1.
I20260924 00:21:00.251003 10370 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet ffa011cf8dc446de849bea75a4f20056
I20260924 00:21:00.252187 10370 catalog_manager.cc:6565] ReplaceTablet: tablet ffa011cf8dc446de849bea75a4f20056 deleted and replaced by tablet 2499754460e04c27ac53d52a4a67f557
I20260924 00:21:00.253734 10773 tablet_service.cc:1514] Processing DeleteTablet for tablet ffa011cf8dc446de849bea75a4f20056 with delete_type TABLET_DATA_DELETED (replaced by tablet 2499754460e04c27ac53d52a4a67f557) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:00.254007 10732 leader_election.cc:302] T ac559183fbef4e21b190a9a61a50e1e1 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters: b10a9082a37b46f09d581944914c1670
I20260924 00:21:00.256451 10628 tablet_service.cc:1514] Processing DeleteTablet for tablet ffa011cf8dc446de849bea75a4f20056 with delete_type TABLET_DATA_DELETED (replaced by tablet 2499754460e04c27ac53d52a4a67f557) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:00.261698 10914 tablet_service.cc:1514] Processing DeleteTablet for tablet ffa011cf8dc446de849bea75a4f20056 with delete_type TABLET_DATA_DELETED (replaced by tablet 2499754460e04c27ac53d52a4a67f557) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:00.262626 10921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf144ab1a6464a4ab02ac0642c5255e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:00.263909 10919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4420cead1f284af8958f189557833098. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:00.265059 10920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95a7a1d2147c4d7ea63ca37a8f0ea4e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:00.266172 10915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4325b60efc1649c9ba4828d0a098a676. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:00.266207 11197 raft_consensus.cc:2781] T ac559183fbef4e21b190a9a61a50e1e1 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:00.266660 11197 raft_consensus.cc:686] T ac559183fbef4e21b190a9a61a50e1e1 P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: Becoming Leader. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Running, Role: LEADER
I20260924 00:21:00.267373 10922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18abdec265a24dc8bfc765d4395767c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:00.273407 11146 raft_consensus.cc:482] T ccb26e0fbc1948bc9a00e9e26c4d1139 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:00.273855 11146 raft_consensus.cc:504] T ccb26e0fbc1948bc9a00e9e26c4d1139 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:00.275832 11146 leader_election.cc:288] T ccb26e0fbc1948bc9a00e9e26c4d1139 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:21:00.278159 10799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccb26e0fbc1948bc9a00e9e26c4d1139" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:00.279764 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccb26e0fbc1948bc9a00e9e26c4d1139" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:00.280347 10942 raft_consensus.cc:2454] T ccb26e0fbc1948bc9a00e9e26c4d1139 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b10a9082a37b46f09d581944914c1670 in term 0.
I20260924 00:21:00.281215 10589 leader_election.cc:302] T ccb26e0fbc1948bc9a00e9e26c4d1139 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
W20260924 00:21:00.283725 10592 leader_election.cc:341] T ccb26e0fbc1948bc9a00e9e26c4d1139 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:00.283882 11146 raft_consensus.cc:2781] T ccb26e0fbc1948bc9a00e9e26c4d1139 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:00.284201 11146 raft_consensus.cc:482] T ccb26e0fbc1948bc9a00e9e26c4d1139 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:00.284468 11146 raft_consensus.cc:3037] T ccb26e0fbc1948bc9a00e9e26c4d1139 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:00.289793 11197 consensus_queue.cc:237] T ac559183fbef4e21b190a9a61a50e1e1 P 48fbd603abe146c585d15196e1b5be08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:00.242125 11040 raft_consensus.cc:374] T 453ce17089ce44ba8da67a0893301c3c P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:00.298157 11040 raft_consensus.cc:729] T 453ce17089ce44ba8da67a0893301c3c P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:00.302785 11146 raft_consensus.cc:504] T ccb26e0fbc1948bc9a00e9e26c4d1139 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:00.303901 11040 consensus_queue.cc:260] T 453ce17089ce44ba8da67a0893301c3c P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:00.307802 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccb26e0fbc1948bc9a00e9e26c4d1139" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef"
I20260924 00:21:00.308518 10942 raft_consensus.cc:3037] T ccb26e0fbc1948bc9a00e9e26c4d1139 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:00.311559 11039 tablet_bootstrap.cc:654] T e0074b1184ec4883ab1016330d98c968 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:00.313752 11198 raft_consensus.cc:482] T bcd8a6b92b8e492d96994195fa7f3ae5 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:00.314906 11041 tablet_bootstrap.cc:492] T 1ec36d59b5114feea144bcb72794136c P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:00.314170 11198 raft_consensus.cc:504] T bcd8a6b92b8e492d96994195fa7f3ae5 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:00.315276 11041 ts_tablet_manager.cc:1397] T 1ec36d59b5114feea144bcb72794136c P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.120s user 0.004s sys 0.006s
I20260924 00:21:00.315568 11040 ts_tablet_manager.cc:1428] T 453ce17089ce44ba8da67a0893301c3c P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.076s user 0.006s sys 0.001s
I20260924 00:21:00.316481 11198 leader_election.cc:288] T bcd8a6b92b8e492d96994195fa7f3ae5 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:21:00.316977 11146 leader_election.cc:288] T ccb26e0fbc1948bc9a00e9e26c4d1139 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Requested vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:21:00.316733 11156 ts_tablet_manager.cc:1918] T 6ad838c884fc4b1d8f98781e7190e3df P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:00.317899 11040 tablet_bootstrap.cc:492] T c212525586544d8f9d92bc4b8bd8acaa P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:00.317350 11041 raft_consensus.cc:348] T 1ec36d59b5114feea144bcb72794136c P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:00.322815 11041 raft_consensus.cc:374] T 1ec36d59b5114feea144bcb72794136c P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:00.323079 11041 raft_consensus.cc:729] T 1ec36d59b5114feea144bcb72794136c P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:00.323168 11156 log.cc:1196] T 6ad838c884fc4b1d8f98781e7190e3df P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/6ad838c884fc4b1d8f98781e7190e3df
I20260924 00:21:00.323623 11041 consensus_queue.cc:260] T 1ec36d59b5114feea144bcb72794136c P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:00.325408 10941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcd8a6b92b8e492d96994195fa7f3ae5" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:00.325585 11041 ts_tablet_manager.cc:1428] T 1ec36d59b5114feea144bcb72794136c P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.010s user 0.006s sys 0.001s
I20260924 00:21:00.326469 11041 tablet_bootstrap.cc:492] T f294831873d548e4b9201c1bbb964096 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
W20260924 00:21:00.329249 10589 leader_election.cc:341] T bcd8a6b92b8e492d96994195fa7f3ae5 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:00.332502 10799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcd8a6b92b8e492d96994195fa7f3ae5" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:00.333828 10800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccb26e0fbc1948bc9a00e9e26c4d1139" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:21:00.334848 11041 tablet_bootstrap.cc:654] T f294831873d548e4b9201c1bbb964096 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
W20260924 00:21:00.340060 10592 leader_election.cc:341] T ccb26e0fbc1948bc9a00e9e26c4d1139 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
W20260924 00:21:00.341447 10592 leader_election.cc:341] T bcd8a6b92b8e492d96994195fa7f3ae5 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:00.341889 10592 leader_election.cc:302] T bcd8a6b92b8e492d96994195fa7f3ae5 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:00.342674 11146 raft_consensus.cc:2726] T bcd8a6b92b8e492d96994195fa7f3ae5 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:00.346509 10942 raft_consensus.cc:2454] T ccb26e0fbc1948bc9a00e9e26c4d1139 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b10a9082a37b46f09d581944914c1670 in term 1.
I20260924 00:21:00.347714 10589 leader_election.cc:302] T ccb26e0fbc1948bc9a00e9e26c4d1139 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08
W20260924 00:21:00.348205 10361 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.124:35345: connect: Connection refused (error 111) [suppressed 9 similar messages]
I20260924 00:21:00.348812 11146 raft_consensus.cc:2781] T ccb26e0fbc1948bc9a00e9e26c4d1139 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:00.349285 11146 raft_consensus.cc:686] T ccb26e0fbc1948bc9a00e9e26c4d1139 P b10a9082a37b46f09d581944914c1670 [term 1 LEADER]: Becoming Leader. State: Replica: b10a9082a37b46f09d581944914c1670, State: Running, Role: LEADER
I20260924 00:21:00.350028 11146 consensus_queue.cc:237] T ccb26e0fbc1948bc9a00e9e26c4d1139 P b10a9082a37b46f09d581944914c1670 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:00.356118 11051 ts_tablet_manager.cc:1918] T 1877e10d1db845f2a001063fb3bffef4 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:00.356483 11051 log.cc:1196] T 1877e10d1db845f2a001063fb3bffef4 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/1877e10d1db845f2a001063fb3bffef4
W20260924 00:21:00.357880 10361 consensus_peers.cc:487] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 -> Peer f8bd0b3184f84f5f8da10c9c21472f92 (127.10.4.124:35345): Couldn't send request to peer f8bd0b3184f84f5f8da10c9c21472f92. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.124:35345: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260924 00:21:00.380074 11051 ts_tablet_manager.cc:1939] T 1877e10d1db845f2a001063fb3bffef4 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:00.381995 11156 ts_tablet_manager.cc:1939] T 6ad838c884fc4b1d8f98781e7190e3df P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:00.387090 11051 tablet_replica.cc:330] T 6ad838c884fc4b1d8f98781e7190e3df P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:00.387349 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 1877e10d1db845f2a001063fb3bffef4 (table default.Table_5f8c295237ab4e0ea184a03a9fa458a2 [id=be698d8b82734ff5940f6915ba1d4df1]) successfully deleted
I20260924 00:21:00.387774 11051 raft_consensus.cc:2229] T 6ad838c884fc4b1d8f98781e7190e3df P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:00.388162 11051 raft_consensus.cc:2258] T 6ad838c884fc4b1d8f98781e7190e3df P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:00.388829 11039 tablet_bootstrap.cc:492] T e0074b1184ec4883ab1016330d98c968 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:00.389220 11039 ts_tablet_manager.cc:1397] T e0074b1184ec4883ab1016330d98c968 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.202s user 0.006s sys 0.003s
I20260924 00:21:00.389174 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 6ad838c884fc4b1d8f98781e7190e3df (table default.Table_2d4af2cf18544b6e94ef78ac7dd3bcce [id=218c77bc322a488593d12af9a27d20a3]) successfully deleted
I20260924 00:21:00.391592 10368 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 6ac9c7da7fb546f6ae935371799cbad6
I20260924 00:21:00.391790 11039 raft_consensus.cc:348] T e0074b1184ec4883ab1016330d98c968 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:00.392396 11039 raft_consensus.cc:374] T e0074b1184ec4883ab1016330d98c968 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:00.392624 11039 raft_consensus.cc:729] T e0074b1184ec4883ab1016330d98c968 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:00.393340 10775 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ac9c7da7fb546f6ae935371799cbad6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:59 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:00.394836 11040 tablet_bootstrap.cc:654] T c212525586544d8f9d92bc4b8bd8acaa P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:00.395484 10370 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 1ec36d59b5114feea144bcb72794136c (table default.Table_b1726c09065e4fbaa2e6799b3ebdbc74 [id=4dbb68e0a32349ad8da2945db650584d]). Expected version 1 got 0
I20260924 00:21:00.396001 11156 tablet_replica.cc:330] T ac559183fbef4e21b190a9a61a50e1e1 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:00.401391 11051 ts_tablet_manager.cc:1905] T 6ad838c884fc4b1d8f98781e7190e3df P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:00.401683 10372 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_41d1376556a9466fa348723fd8e2b2be" } modify_external_catalogs: true
I20260924 00:21:00.402282 10372 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_41d1376556a9466fa348723fd8e2b2be" } modify_external_catalogs: true
W20260924 00:21:00.404376 10370 catalog_manager.cc:5687] Failed to send AlterTable RPC for tablet 1ec36d59b5114feea144bcb72794136c (table default.Table_b1726c09065e4fbaa2e6799b3ebdbc74 [id=4dbb68e0a32349ad8da2945db650584d], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 1ec36d59b5114feea144bcb72794136c consensus state has no leader
I20260924 00:21:00.403239 11039 consensus_queue.cc:260] T e0074b1184ec4883ab1016330d98c968 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:00.407848 10919 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ac9c7da7fb546f6ae935371799cbad6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:59 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:00.412111 11039 ts_tablet_manager.cc:1428] T e0074b1184ec4883ab1016330d98c968 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.023s user 0.005s sys 0.000s
I20260924 00:21:00.417086 11199 raft_consensus.cc:482] T 453ce17089ce44ba8da67a0893301c3c P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:00.417567 11199 raft_consensus.cc:504] T 453ce17089ce44ba8da67a0893301c3c P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:00.419582 11199 leader_election.cc:288] T 453ce17089ce44ba8da67a0893301c3c P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:00.419942 11039 tablet_bootstrap.cc:492] T ae0c9b785195491987dd8fe397d704e0 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
W20260924 00:21:00.421499 10733 leader_election.cc:341] T 453ce17089ce44ba8da67a0893301c3c P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: 453ce17089ce44ba8da67a0893301c3c
I20260924 00:21:00.422683 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "453ce17089ce44ba8da67a0893301c3c" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:00.423236 10942 raft_consensus.cc:2454] T 453ce17089ce44ba8da67a0893301c3c P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 0.
I20260924 00:21:00.424397 10732 leader_election.cc:302] T 453ce17089ce44ba8da67a0893301c3c P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters: b10a9082a37b46f09d581944914c1670
I20260924 00:21:00.425109 11039 tablet_bootstrap.cc:654] T ae0c9b785195491987dd8fe397d704e0 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:00.425439 11199 raft_consensus.cc:2781] T 453ce17089ce44ba8da67a0893301c3c P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:00.425781 11199 raft_consensus.cc:482] T 453ce17089ce44ba8da67a0893301c3c P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:00.426179 11199 raft_consensus.cc:3037] T 453ce17089ce44ba8da67a0893301c3c P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:00.426131 10368 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet e0074b1184ec4883ab1016330d98c968 (table default.Table_63927aa1872e41da917d6b92c8acb310 [id=485c42c942b74abbb32bea36d25bc184]). Expected version 2 got 0
I20260924 00:21:00.433277 11199 raft_consensus.cc:504] T 453ce17089ce44ba8da67a0893301c3c P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:00.434335 10368 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet e0074b1184ec4883ab1016330d98c968 (table default.Table_63927aa1872e41da917d6b92c8acb310 [id=485c42c942b74abbb32bea36d25bc184]). Expected version 2 got 0
I20260924 00:21:00.435920 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "453ce17089ce44ba8da67a0893301c3c" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef"
I20260924 00:21:00.436426 10942 raft_consensus.cc:3037] T 453ce17089ce44ba8da67a0893301c3c P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:00.440785 11039 tablet_bootstrap.cc:492] T ae0c9b785195491987dd8fe397d704e0 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:00.443341 10942 raft_consensus.cc:2454] T 453ce17089ce44ba8da67a0893301c3c P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 1.
I20260924 00:21:00.444285 11039 ts_tablet_manager.cc:1397] T ae0c9b785195491987dd8fe397d704e0 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.025s user 0.000s sys 0.009s
I20260924 00:21:00.444456 10732 leader_election.cc:302] T 453ce17089ce44ba8da67a0893301c3c P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:00.445747 11150 raft_consensus.cc:2781] T 453ce17089ce44ba8da67a0893301c3c P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:00.446941 11039 raft_consensus.cc:348] T ae0c9b785195491987dd8fe397d704e0 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:00.447536 11039 raft_consensus.cc:374] T ae0c9b785195491987dd8fe397d704e0 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:00.447757 11039 raft_consensus.cc:729] T ae0c9b785195491987dd8fe397d704e0 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:00.457612 11150 raft_consensus.cc:686] T 453ce17089ce44ba8da67a0893301c3c P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: Becoming Leader. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Running, Role: LEADER
I20260924 00:21:00.459100 11150 consensus_queue.cc:237] T 453ce17089ce44ba8da67a0893301c3c P 48fbd603abe146c585d15196e1b5be08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:00.456293 11039 consensus_queue.cc:260] T ae0c9b785195491987dd8fe397d704e0 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:00.462257 11039 ts_tablet_manager.cc:1428] T ae0c9b785195491987dd8fe397d704e0 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.018s user 0.002s sys 0.003s
I20260924 00:21:00.463013 11039 tablet_bootstrap.cc:492] T c212525586544d8f9d92bc4b8bd8acaa P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
W20260924 00:21:00.464957 10733 leader_election.cc:341] T 453ce17089ce44ba8da67a0893301c3c P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: 453ce17089ce44ba8da67a0893301c3c
W20260924 00:21:00.465611 10361 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 1ec36d59b5114feea144bcb72794136c (table default.Table_b1726c09065e4fbaa2e6799b3ebdbc74 [id=4dbb68e0a32349ad8da2945db650584d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 1ec36d59b5114feea144bcb72794136c consensus state has no leader
I20260924 00:21:00.466887 11040 tablet_bootstrap.cc:492] T c212525586544d8f9d92bc4b8bd8acaa P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:00.467172 11199 leader_election.cc:288] T 453ce17089ce44ba8da67a0893301c3c P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Requested vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:00.476466 11039 tablet_bootstrap.cc:654] T c212525586544d8f9d92bc4b8bd8acaa P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
W20260924 00:21:00.480319 10373 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4420cead1f284af8958f189557833098 consensus state has no leader
I20260924 00:21:00.487332 11156 raft_consensus.cc:2229] T ac559183fbef4e21b190a9a61a50e1e1 P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:21:00.492795 11039 tablet_bootstrap.cc:492] T c212525586544d8f9d92bc4b8bd8acaa P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:00.493193 11039 ts_tablet_manager.cc:1397] T c212525586544d8f9d92bc4b8bd8acaa P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.030s user 0.009s sys 0.000s
I20260924 00:21:00.495563 11156 pending_rounds.cc:62] T ac559183fbef4e21b190a9a61a50e1e1 P 48fbd603abe146c585d15196e1b5be08: Trying to abort 1 pending ops.
I20260924 00:21:00.495821 11156 pending_rounds.cc:68] T ac559183fbef4e21b190a9a61a50e1e1 P 48fbd603abe146c585d15196e1b5be08: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332697130157707264 op_type: NO_OP noop_request { }
I20260924 00:21:00.496731 11156 raft_consensus.cc:2866] T ac559183fbef4e21b190a9a61a50e1e1 P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260924 00:21:00.500830 11156 raft_consensus.cc:2258] T ac559183fbef4e21b190a9a61a50e1e1 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:00.498056 11040 ts_tablet_manager.cc:1397] T c212525586544d8f9d92bc4b8bd8acaa P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.180s user 0.014s sys 0.000s
I20260924 00:21:00.495155 11039 raft_consensus.cc:348] T c212525586544d8f9d92bc4b8bd8acaa P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:00.503219 11039 raft_consensus.cc:374] T c212525586544d8f9d92bc4b8bd8acaa P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:00.503481 11039 raft_consensus.cc:729] T c212525586544d8f9d92bc4b8bd8acaa P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:00.503763 11040 raft_consensus.cc:348] T c212525586544d8f9d92bc4b8bd8acaa P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:00.504022 11039 consensus_queue.cc:260] T c212525586544d8f9d92bc4b8bd8acaa P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:00.504428 11040 raft_consensus.cc:374] T c212525586544d8f9d92bc4b8bd8acaa P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:00.504652 11040 raft_consensus.cc:729] T c212525586544d8f9d92bc4b8bd8acaa P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:00.505916 11039 ts_tablet_manager.cc:1428] T c212525586544d8f9d92bc4b8bd8acaa P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.012s user 0.005s sys 0.000s
I20260924 00:21:00.506588 11051 ts_tablet_manager.cc:1918] T 6ad838c884fc4b1d8f98781e7190e3df P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:00.506328 11040 consensus_queue.cc:260] T c212525586544d8f9d92bc4b8bd8acaa P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:00.506911 11051 log.cc:1196] T 6ad838c884fc4b1d8f98781e7190e3df P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/6ad838c884fc4b1d8f98781e7190e3df
I20260924 00:21:00.507483 11051 ts_tablet_manager.cc:1939] T 6ad838c884fc4b1d8f98781e7190e3df P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:00.397979 10632 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ac9c7da7fb546f6ae935371799cbad6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:59 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:00.510911 10373 master_service.cc:946] ReplaceTablet: received request to replace tablet 95a7a1d2147c4d7ea63ca37a8f0ea4e6 from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:00.514935 11040 ts_tablet_manager.cc:1428] T c212525586544d8f9d92bc4b8bd8acaa P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.013s user 0.001s sys 0.004s
I20260924 00:21:00.524125 11040 tablet_bootstrap.cc:492] T ccb26e0fbc1948bc9a00e9e26c4d1139 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
W20260924 00:21:00.539584 11206 consensus_queue.cc:1159] T ac559183fbef4e21b190a9a61a50e1e1 P 48fbd603abe146c585d15196e1b5be08 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "48fbd603abe146c585d15196e1b5be08" status { last_received { term: 1 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 1 } }
I20260924 00:21:00.544237 11156 ts_tablet_manager.cc:1905] T ac559183fbef4e21b190a9a61a50e1e1 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:00.378139 11049 ts_tablet_manager.cc:1918] T c212525586544d8f9d92bc4b8bd8acaa P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:00.551923 11039 tablet_bootstrap.cc:492] T af2f58e6757a4b738a8ec38b2e228f9f P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:00.552042 11049 log.cc:1196] T c212525586544d8f9d92bc4b8bd8acaa P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/c212525586544d8f9d92bc4b8bd8acaa
I20260924 00:21:00.555485 11049 ts_tablet_manager.cc:1939] T c212525586544d8f9d92bc4b8bd8acaa P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:00.556366 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 6ad838c884fc4b1d8f98781e7190e3df (table default.Table_2d4af2cf18544b6e94ef78ac7dd3bcce [id=218c77bc322a488593d12af9a27d20a3]) successfully deleted
I20260924 00:21:00.556594 11039 tablet_bootstrap.cc:654] T af2f58e6757a4b738a8ec38b2e228f9f P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
W20260924 00:21:00.557469 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet b06bf1e4691441769e162aa5404d4656 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b06bf1e4691441769e162aa5404d4656 already in progress: creating tablet
I20260924 00:21:00.558413 11040 tablet_bootstrap.cc:654] T ccb26e0fbc1948bc9a00e9e26c4d1139 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:00.558616 11051 tablet_replica.cc:330] T ac559183fbef4e21b190a9a61a50e1e1 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:00.559239 11051 raft_consensus.cc:2229] T ac559183fbef4e21b190a9a61a50e1e1 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:00.559731 11051 raft_consensus.cc:2258] T ac559183fbef4e21b190a9a61a50e1e1 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 00:21:00.559626 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 2964cd68d5e94f3780a6a3157bd86983 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2964cd68d5e94f3780a6a3157bd86983 already in progress: creating tablet
I20260924 00:21:00.562882 11151 raft_consensus.cc:482] T e0074b1184ec4883ab1016330d98c968 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:00.563326 11151 raft_consensus.cc:504] T e0074b1184ec4883ab1016330d98c968 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:00.564858 11051 ts_tablet_manager.cc:1905] T ac559183fbef4e21b190a9a61a50e1e1 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:00.565748 11156 ts_tablet_manager.cc:1918] T ac559183fbef4e21b190a9a61a50e1e1 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:00.566118 11156 log.cc:1196] T ac559183fbef4e21b190a9a61a50e1e1 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/ac559183fbef4e21b190a9a61a50e1e1
I20260924 00:21:00.566179 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0074b1184ec4883ab1016330d98c968" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:00.566848 11156 ts_tablet_manager.cc:1939] T ac559183fbef4e21b190a9a61a50e1e1 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:00.566814 10654 raft_consensus.cc:2454] T e0074b1184ec4883ab1016330d98c968 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
I20260924 00:21:00.568068 10876 leader_election.cc:302] T e0074b1184ec4883ab1016330d98c968 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:00.569651 11151 leader_election.cc:288] T e0074b1184ec4883ab1016330d98c968 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:21:00.570086 11151 raft_consensus.cc:2781] T e0074b1184ec4883ab1016330d98c968 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:00.570387 11151 raft_consensus.cc:482] T e0074b1184ec4883ab1016330d98c968 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:00.570662 11151 raft_consensus.cc:3037] T e0074b1184ec4883ab1016330d98c968 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:00.573310 11040 tablet_bootstrap.cc:492] T ccb26e0fbc1948bc9a00e9e26c4d1139 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:00.573721 11040 ts_tablet_manager.cc:1397] T ccb26e0fbc1948bc9a00e9e26c4d1139 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.050s user 0.006s sys 0.006s
I20260924 00:21:00.576017 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet c212525586544d8f9d92bc4b8bd8acaa (table default.Table_ef16ff2bfc9b42219f0b6d5f957312f6 [id=fbdec8eccace4eb3864a6cdd95795ce3]) successfully deleted
I20260924 00:21:00.575822 11040 raft_consensus.cc:348] T ccb26e0fbc1948bc9a00e9e26c4d1139 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:00.576426 11040 raft_consensus.cc:374] T ccb26e0fbc1948bc9a00e9e26c4d1139 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:00.576645 11040 raft_consensus.cc:729] T ccb26e0fbc1948bc9a00e9e26c4d1139 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:00.576813 11049 tablet_replica.cc:330] T bcd8a6b92b8e492d96994195fa7f3ae5 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:00.579241 10799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0074b1184ec4883ab1016330d98c968" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:00.581207 11151 raft_consensus.cc:504] T e0074b1184ec4883ab1016330d98c968 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:00.581707 10366 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet ae0c9b785195491987dd8fe397d704e0 (table default.Table_d61af65b53ce4871b7b24752281af745 [id=0edc6c93a1ba4c8d81755129301c83de]). Expected version 1 got 0
I20260924 00:21:00.582405 11049 raft_consensus.cc:2229] T bcd8a6b92b8e492d96994195fa7f3ae5 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:00.582932 11151 leader_election.cc:288] T e0074b1184ec4883ab1016330d98c968 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:21:00.584157 10800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0074b1184ec4883ab1016330d98c968" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:21:00.585616 11049 raft_consensus.cc:2258] T bcd8a6b92b8e492d96994195fa7f3ae5 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:00.590073 10780 tablet_service.cc:1514] Processing DeleteTablet for tablet c212525586544d8f9d92bc4b8bd8acaa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:57 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:00.591773 10776 tablet_service.cc:1514] Processing DeleteTablet for tablet 453ce17089ce44ba8da67a0893301c3c with delete_type TABLET_DATA_DELETED (replaced by tablet 195b033091b543b58ee08417eae2c0e6) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:00.593148 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet ac559183fbef4e21b190a9a61a50e1e1 (table default.Table_3b2077a1a0a446f98a6ddb235c703345 [id=8063a759be2141ae965703f9196a18c8]) successfully deleted
I20260924 00:21:00.594048 11156 tablet_replica.cc:330] T c83942a74816465fa7f3f2c3e18c98bb P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:00.595489 10365 master_service.cc:946] ReplaceTablet: received request to replace tablet 5e624be739394e6d87d6b3fef2f46302 from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:00.599757 11156 raft_consensus.cc:2229] T c83942a74816465fa7f3f2c3e18c98bb P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:00.602593 11039 tablet_bootstrap.cc:492] T af2f58e6757a4b738a8ec38b2e228f9f P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:00.602994 11039 ts_tablet_manager.cc:1397] T af2f58e6757a4b738a8ec38b2e228f9f P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.051s user 0.007s sys 0.007s
I20260924 00:21:00.605114 11039 raft_consensus.cc:348] T af2f58e6757a4b738a8ec38b2e228f9f P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:00.605757 11039 raft_consensus.cc:374] T af2f58e6757a4b738a8ec38b2e228f9f P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:00.606007 11039 raft_consensus.cc:729] T af2f58e6757a4b738a8ec38b2e228f9f P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:00.605461 10921 tablet_service.cc:1514] Processing DeleteTablet for tablet c212525586544d8f9d92bc4b8bd8acaa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:57 UTC) from {username='slave'} at 127.0.0.1:35358
W20260924 00:21:00.605469 10878 leader_election.cc:341] T e0074b1184ec4883ab1016330d98c968 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:00.585104 11146 raft_consensus.cc:482] T ae0c9b785195491987dd8fe397d704e0 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:00.606937 11146 raft_consensus.cc:504] T ae0c9b785195491987dd8fe397d704e0 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:00.608741 11146 leader_election.cc:288] T ae0c9b785195491987dd8fe397d704e0 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:00.595891 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0074b1184ec4883ab1016330d98c968" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:21:00.609287 10654 raft_consensus.cc:3037] T e0074b1184ec4883ab1016330d98c968 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
W20260924 00:21:00.609669 10878 leader_election.cc:341] T e0074b1184ec4883ab1016330d98c968 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:00.611223 11156 raft_consensus.cc:2258] T c83942a74816465fa7f3f2c3e18c98bb P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:00.614065 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae0c9b785195491987dd8fe397d704e0" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:00.614665 10942 raft_consensus.cc:2454] T ae0c9b785195491987dd8fe397d704e0 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b10a9082a37b46f09d581944914c1670 in term 0.
I20260924 00:21:00.606581 11039 consensus_queue.cc:260] T af2f58e6757a4b738a8ec38b2e228f9f P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:00.616962 10589 leader_election.cc:302] T ae0c9b785195491987dd8fe397d704e0 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:00.617338 10654 raft_consensus.cc:2454] T e0074b1184ec4883ab1016330d98c968 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 1.
I20260924 00:21:00.618090 10800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae0c9b785195491987dd8fe397d704e0" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:00.618347 10876 leader_election.cc:302] T e0074b1184ec4883ab1016330d98c968 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08
W20260924 00:21:00.619499 10592 leader_election.cc:341] T ae0c9b785195491987dd8fe397d704e0 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:00.619756 11218 raft_consensus.cc:2781] T e0074b1184ec4883ab1016330d98c968 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:00.620342 11146 raft_consensus.cc:2781] T ae0c9b785195491987dd8fe397d704e0 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:00.620648 11146 raft_consensus.cc:482] T ae0c9b785195491987dd8fe397d704e0 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:00.620931 11146 raft_consensus.cc:3037] T ae0c9b785195491987dd8fe397d704e0 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:00.620990 11051 ts_tablet_manager.cc:1918] T ac559183fbef4e21b190a9a61a50e1e1 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:00.621297 11051 log.cc:1196] T ac559183fbef4e21b190a9a61a50e1e1 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/ac559183fbef4e21b190a9a61a50e1e1
I20260924 00:21:00.621920 11051 ts_tablet_manager.cc:1939] T ac559183fbef4e21b190a9a61a50e1e1 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:00.622972 11039 ts_tablet_manager.cc:1428] T af2f58e6757a4b738a8ec38b2e228f9f P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.020s user 0.005s sys 0.000s
I20260924 00:21:00.623791 11039 tablet_bootstrap.cc:492] T eeb11092b803467e9db732468350e852 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:00.623871 11051 tablet_replica.cc:330] T c83942a74816465fa7f3f2c3e18c98bb P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:00.627902 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet ac559183fbef4e21b190a9a61a50e1e1 (table default.Table_3b2077a1a0a446f98a6ddb235c703345 [id=8063a759be2141ae965703f9196a18c8]) successfully deleted
I20260924 00:21:00.627979 10366 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_d61af65b53ce4871b7b24752281af745" } modify_external_catalogs: true
I20260924 00:21:00.628520 10366 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_d61af65b53ce4871b7b24752281af745" } modify_external_catalogs: true
I20260924 00:21:00.630363 11156 ts_tablet_manager.cc:1905] T c83942a74816465fa7f3f2c3e18c98bb P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:00.632295 11049 ts_tablet_manager.cc:1905] T bcd8a6b92b8e492d96994195fa7f3ae5 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:00.629537 11146 raft_consensus.cc:504] T ae0c9b785195491987dd8fe397d704e0 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:00.632647 10921 tablet_service.cc:1514] Processing DeleteTablet for tablet af2f58e6757a4b738a8ec38b2e228f9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:57 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:00.611981 11040 consensus_queue.cc:260] T ccb26e0fbc1948bc9a00e9e26c4d1139 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:00.641251 11040 ts_tablet_manager.cc:1428] T ccb26e0fbc1948bc9a00e9e26c4d1139 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.067s user 0.007s sys 0.001s
I20260924 00:21:00.644404 11039 tablet_bootstrap.cc:654] T eeb11092b803467e9db732468350e852 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:00.644850 11040 tablet_bootstrap.cc:492] T e0074b1184ec4883ab1016330d98c968 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:00.632225 11041 tablet_bootstrap.cc:492] T f294831873d548e4b9201c1bbb964096 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:00.645577 11041 ts_tablet_manager.cc:1397] T f294831873d548e4b9201c1bbb964096 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.319s user 0.041s sys 0.056s
I20260924 00:21:00.647374 10800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae0c9b785195491987dd8fe397d704e0" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:21:00.647809 11041 raft_consensus.cc:348] T f294831873d548e4b9201c1bbb964096 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:00.648418 11041 raft_consensus.cc:374] T f294831873d548e4b9201c1bbb964096 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:00.648643 11041 raft_consensus.cc:729] T f294831873d548e4b9201c1bbb964096 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
W20260924 00:21:00.649456 10592 leader_election.cc:341] T ae0c9b785195491987dd8fe397d704e0 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:00.649240 11041 consensus_queue.cc:260] T f294831873d548e4b9201c1bbb964096 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:00.651458 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae0c9b785195491987dd8fe397d704e0" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef"
I20260924 00:21:00.652030 10942 raft_consensus.cc:3037] T ae0c9b785195491987dd8fe397d704e0 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:00.653959 11041 ts_tablet_manager.cc:1428] T f294831873d548e4b9201c1bbb964096 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20260924 00:21:00.654717 11041 tablet_bootstrap.cc:492] T 5d7d52be4f1f4fc29b8172b9fa878f76 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:00.657965 11150 raft_consensus.cc:482] T c212525586544d8f9d92bc4b8bd8acaa P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:00.658406 11150 raft_consensus.cc:504] T c212525586544d8f9d92bc4b8bd8acaa P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:00.660333 11150 leader_election.cc:288] T c212525586544d8f9d92bc4b8bd8acaa P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:00.660430 11198 raft_consensus.cc:482] T f294831873d548e4b9201c1bbb964096 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:00.660792 11198 raft_consensus.cc:504] T f294831873d548e4b9201c1bbb964096 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:00.661202 10368 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet ccb26e0fbc1948bc9a00e9e26c4d1139 (table default.Table_01b5209b9d5c4115b0bf6aef48b51bd8 [id=7e9e663acedf4a1fae2f04cde97b7907]). Expected version 1 got 0
I20260924 00:21:00.661733 10941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c212525586544d8f9d92bc4b8bd8acaa" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:00.662348 10941 raft_consensus.cc:2454] T c212525586544d8f9d92bc4b8bd8acaa P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 0.
I20260924 00:21:00.662513 11198 leader_election.cc:288] T f294831873d548e4b9201c1bbb964096 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:00.663121 10942 raft_consensus.cc:2454] T ae0c9b785195491987dd8fe397d704e0 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b10a9082a37b46f09d581944914c1670 in term 1.
I20260924 00:21:00.663249 10732 leader_election.cc:302] T c212525586544d8f9d92bc4b8bd8acaa P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:00.663609 10371 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_13a88cf182fe497d84366126f2fdd4ee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:00.664160 11150 raft_consensus.cc:2781] T c212525586544d8f9d92bc4b8bd8acaa P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:00.664300 10589 leader_election.cc:302] T ae0c9b785195491987dd8fe397d704e0 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08
I20260924 00:21:00.664544 11150 raft_consensus.cc:482] T c212525586544d8f9d92bc4b8bd8acaa P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:00.664827 11150 raft_consensus.cc:3037] T c212525586544d8f9d92bc4b8bd8acaa P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
W20260924 00:21:00.665766 10371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_13a88cf182fe497d84366126f2fdd4ee in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:00.666074 11198 raft_consensus.cc:2781] T ae0c9b785195491987dd8fe397d704e0 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:00.666443 10800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f294831873d548e4b9201c1bbb964096" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:00.668324 10592 leader_election.cc:341] T f294831873d548e4b9201c1bbb964096 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:00.669242 11146 leader_election.cc:288] T ae0c9b785195491987dd8fe397d704e0 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Requested vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
W20260924 00:21:00.674989 10733 leader_election.cc:341] T c212525586544d8f9d92bc4b8bd8acaa P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: c212525586544d8f9d92bc4b8bd8acaa
I20260924 00:21:00.675997 11198 raft_consensus.cc:686] T ae0c9b785195491987dd8fe397d704e0 P b10a9082a37b46f09d581944914c1670 [term 1 LEADER]: Becoming Leader. State: Replica: b10a9082a37b46f09d581944914c1670, State: Running, Role: LEADER
I20260924 00:21:00.676779 11198 consensus_queue.cc:237] T ae0c9b785195491987dd8fe397d704e0 P b10a9082a37b46f09d581944914c1670 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:00.680565 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f294831873d548e4b9201c1bbb964096" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
W20260924 00:21:00.681770 10589 leader_election.cc:341] T f294831873d548e4b9201c1bbb964096 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:00.682161 10589 leader_election.cc:302] T f294831873d548e4b9201c1bbb964096 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:00.678859 11150 raft_consensus.cc:504] T c212525586544d8f9d92bc4b8bd8acaa P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:00.685724 11220 raft_consensus.cc:2726] T f294831873d548e4b9201c1bbb964096 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:00.686575 11040 tablet_bootstrap.cc:654] T e0074b1184ec4883ab1016330d98c968 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:00.687079 11051 raft_consensus.cc:2229] T c83942a74816465fa7f3f2c3e18c98bb P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:00.689801 11051 raft_consensus.cc:2258] T c83942a74816465fa7f3f2c3e18c98bb P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:00.691664 11051 ts_tablet_manager.cc:1905] T c83942a74816465fa7f3f2c3e18c98bb P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:00.695864 11199 raft_consensus.cc:482] T ccb26e0fbc1948bc9a00e9e26c4d1139 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:00.696367 11199 raft_consensus.cc:504] T ccb26e0fbc1948bc9a00e9e26c4d1139 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:00.696715 11041 tablet_bootstrap.cc:654] T 5d7d52be4f1f4fc29b8172b9fa878f76 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:00.698251 11199 leader_election.cc:288] T ccb26e0fbc1948bc9a00e9e26c4d1139 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:00.698987 10368 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_ec06ce66fea04103be3c890fc13493d9" } new_table_name: "default.Table_d952b34e7fe64fe494f56df574e61d77" modify_external_catalogs: true
I20260924 00:21:00.700163 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccb26e0fbc1948bc9a00e9e26c4d1139" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:00.700749 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccb26e0fbc1948bc9a00e9e26c4d1139" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:00.702232 10942 raft_consensus.cc:2379] T ccb26e0fbc1948bc9a00e9e26c4d1139 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48fbd603abe146c585d15196e1b5be08 in current term 1: Already voted for candidate b10a9082a37b46f09d581944914c1670 in this term.
I20260924 00:21:00.702414 10367 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet ccb26e0fbc1948bc9a00e9e26c4d1139 (table default.Table_01b5209b9d5c4115b0bf6aef48b51bd8 [id=7e9e663acedf4a1fae2f04cde97b7907]). Expected version 1 got 0
I20260924 00:21:00.702929 10941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c212525586544d8f9d92bc4b8bd8acaa" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef"
I20260924 00:21:00.703421 10941 raft_consensus.cc:3037] T c212525586544d8f9d92bc4b8bd8acaa P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:00.705439 10732 leader_election.cc:302] T ccb26e0fbc1948bc9a00e9e26c4d1139 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08; no voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:00.706648 11199 raft_consensus.cc:3037] T ccb26e0fbc1948bc9a00e9e26c4d1139 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:00.709123 11041 tablet_bootstrap.cc:492] T 5d7d52be4f1f4fc29b8172b9fa878f76 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:00.709494 11041 ts_tablet_manager.cc:1397] T 5d7d52be4f1f4fc29b8172b9fa878f76 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.055s user 0.007s sys 0.002s
I20260924 00:21:00.716197 11220 raft_consensus.cc:482] T 1ec36d59b5114feea144bcb72794136c P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:00.712322 10941 raft_consensus.cc:2454] T c212525586544d8f9d92bc4b8bd8acaa P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 1.
I20260924 00:21:00.716598 11220 raft_consensus.cc:504] T 1ec36d59b5114feea144bcb72794136c P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:00.718411 11199 raft_consensus.cc:2726] T ccb26e0fbc1948bc9a00e9e26c4d1139 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:00.718477 11220 leader_election.cc:288] T 1ec36d59b5114feea144bcb72794136c P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:00.719385 11040 tablet_bootstrap.cc:492] T e0074b1184ec4883ab1016330d98c968 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:00.719849 11040 ts_tablet_manager.cc:1397] T e0074b1184ec4883ab1016330d98c968 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.075s user 0.018s sys 0.000s
I20260924 00:21:00.720033 10732 leader_election.cc:302] T c212525586544d8f9d92bc4b8bd8acaa P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:00.721012 11199 raft_consensus.cc:2781] T c212525586544d8f9d92bc4b8bd8acaa P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:00.721858 10941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ec36d59b5114feea144bcb72794136c" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:00.722260 10800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ec36d59b5114feea144bcb72794136c" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:00.723107 10589 leader_election.cc:341] T 1ec36d59b5114feea144bcb72794136c P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
W20260924 00:21:00.723506 10592 leader_election.cc:341] T 1ec36d59b5114feea144bcb72794136c P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:00.723846 10592 leader_election.cc:302] T 1ec36d59b5114feea144bcb72794136c P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:00.724865 11198 raft_consensus.cc:2726] T 1ec36d59b5114feea144bcb72794136c P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:00.724617 11040 raft_consensus.cc:348] T e0074b1184ec4883ab1016330d98c968 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:00.725287 11040 raft_consensus.cc:374] T e0074b1184ec4883ab1016330d98c968 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:00.726076 11199 raft_consensus.cc:686] T c212525586544d8f9d92bc4b8bd8acaa P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: Becoming Leader. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Running, Role: LEADER
I20260924 00:21:00.726241 10367 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_0c36f6e4aa264a4394fd0bc65fd035a8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:00.726966 11199 consensus_queue.cc:237] T c212525586544d8f9d92bc4b8bd8acaa P 48fbd603abe146c585d15196e1b5be08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
W20260924 00:21:00.727944 10367 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0c36f6e4aa264a4394fd0bc65fd035a8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:00.731769 11218 raft_consensus.cc:686] T e0074b1184ec4883ab1016330d98c968 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Becoming Leader. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Running, Role: LEADER
I20260924 00:21:00.732525 11218 consensus_queue.cc:237] T e0074b1184ec4883ab1016330d98c968 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:00.725531 11040 raft_consensus.cc:729] T e0074b1184ec4883ab1016330d98c968 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:00.712066 11041 raft_consensus.cc:348] T 5d7d52be4f1f4fc29b8172b9fa878f76 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:00.734303 11040 consensus_queue.cc:260] T e0074b1184ec4883ab1016330d98c968 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:00.734975 11041 raft_consensus.cc:374] T 5d7d52be4f1f4fc29b8172b9fa878f76 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:00.735244 11041 raft_consensus.cc:729] T 5d7d52be4f1f4fc29b8172b9fa878f76 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:00.735934 11041 consensus_queue.cc:260] T 5d7d52be4f1f4fc29b8172b9fa878f76 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:00.739820 11040 ts_tablet_manager.cc:1428] T e0074b1184ec4883ab1016330d98c968 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.020s user 0.005s sys 0.000s
I20260924 00:21:00.744272 11156 ts_tablet_manager.cc:1918] T c83942a74816465fa7f3f2c3e18c98bb P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:00.744699 11156 log.cc:1196] T c83942a74816465fa7f3f2c3e18c98bb P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/c83942a74816465fa7f3f2c3e18c98bb
I20260924 00:21:00.745621 11041 ts_tablet_manager.cc:1428] T 5d7d52be4f1f4fc29b8172b9fa878f76 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.036s user 0.005s sys 0.001s
I20260924 00:21:00.744890 11040 tablet_bootstrap.cc:492] T af2f58e6757a4b738a8ec38b2e228f9f P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:00.746404 11156 ts_tablet_manager.cc:1939] T c83942a74816465fa7f3f2c3e18c98bb P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:00.755228 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet c83942a74816465fa7f3f2c3e18c98bb (table default.Table_41d1376556a9466fa348723fd8e2b2be [id=7eb84ebbe346450eb2b944db685c414b]) successfully deleted
I20260924 00:21:00.755872 11039 tablet_bootstrap.cc:492] T eeb11092b803467e9db732468350e852 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:00.756270 11039 ts_tablet_manager.cc:1397] T eeb11092b803467e9db732468350e852 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.133s user 0.010s sys 0.000s
I20260924 00:21:00.757019 11041 tablet_bootstrap.cc:492] T 86374d72238d42c29542e649a7a0200d P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
W20260924 00:21:00.758345 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet c7e50ac8b7f947b1bfba8cd1cff0ea7e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c7e50ac8b7f947b1bfba8cd1cff0ea7e already in progress: creating tablet
I20260924 00:21:00.758330 11039 raft_consensus.cc:348] T eeb11092b803467e9db732468350e852 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:00.758942 11039 raft_consensus.cc:374] T eeb11092b803467e9db732468350e852 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:00.759182 11039 raft_consensus.cc:729] T eeb11092b803467e9db732468350e852 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:00.760761 11040 tablet_bootstrap.cc:654] T af2f58e6757a4b738a8ec38b2e228f9f P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
W20260924 00:21:00.762080 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet ffa011cf8dc446de849bea75a4f20056 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ffa011cf8dc446de849bea75a4f20056 already in progress: creating tablet
I20260924 00:21:00.762679 11041 tablet_bootstrap.cc:654] T 86374d72238d42c29542e649a7a0200d P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
W20260924 00:21:00.768218 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 6ac9c7da7fb546f6ae935371799cbad6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6ac9c7da7fb546f6ae935371799cbad6 already in progress: creating tablet
W20260924 00:21:00.770975 10369 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f294831873d548e4b9201c1bbb964096 consensus state has no leader
I20260924 00:21:00.772460 11156 tablet_replica.cc:330] T c212525586544d8f9d92bc4b8bd8acaa P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:00.777070 11051 ts_tablet_manager.cc:1918] T c83942a74816465fa7f3f2c3e18c98bb P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:00.777427 11051 log.cc:1196] T c83942a74816465fa7f3f2c3e18c98bb P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/c83942a74816465fa7f3f2c3e18c98bb
I20260924 00:21:00.778015 11051 ts_tablet_manager.cc:1939] T c83942a74816465fa7f3f2c3e18c98bb P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:00.780026 11150 leader_election.cc:288] T c212525586544d8f9d92bc4b8bd8acaa P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Requested vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
W20260924 00:21:00.781428 10733 leader_election.cc:341] T c212525586544d8f9d92bc4b8bd8acaa P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: c212525586544d8f9d92bc4b8bd8acaa
I20260924 00:21:00.785200 11041 tablet_bootstrap.cc:492] T 86374d72238d42c29542e649a7a0200d P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:00.785614 11041 ts_tablet_manager.cc:1397] T 86374d72238d42c29542e649a7a0200d P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.029s user 0.011s sys 0.001s
I20260924 00:21:00.787747 11041 raft_consensus.cc:348] T 86374d72238d42c29542e649a7a0200d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:00.788478 11041 raft_consensus.cc:374] T 86374d72238d42c29542e649a7a0200d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:00.788750 11041 raft_consensus.cc:729] T 86374d72238d42c29542e649a7a0200d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:00.791105 11040 tablet_bootstrap.cc:492] T af2f58e6757a4b738a8ec38b2e228f9f P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:00.791464 11040 ts_tablet_manager.cc:1397] T af2f58e6757a4b738a8ec38b2e228f9f P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.047s user 0.009s sys 0.000s
I20260924 00:21:00.792725 10374 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet e0074b1184ec4883ab1016330d98c968 (table default.Table_63927aa1872e41da917d6b92c8acb310 [id=485c42c942b74abbb32bea36d25bc184]). Expected version 2 got 0
I20260924 00:21:00.795413 11156 raft_consensus.cc:2229] T c212525586544d8f9d92bc4b8bd8acaa P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:21:00.796514 11156 pending_rounds.cc:62] T c212525586544d8f9d92bc4b8bd8acaa P 48fbd603abe146c585d15196e1b5be08: Trying to abort 1 pending ops.
I20260924 00:21:00.796736 11156 pending_rounds.cc:68] T c212525586544d8f9d92bc4b8bd8acaa P 48fbd603abe146c585d15196e1b5be08: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332697131945512960 op_type: NO_OP noop_request { }
I20260924 00:21:00.797088 11156 raft_consensus.cc:2866] T c212525586544d8f9d92bc4b8bd8acaa P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260924 00:21:00.797372 11156 raft_consensus.cc:2258] T c212525586544d8f9d92bc4b8bd8acaa P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:00.797500 11040 raft_consensus.cc:348] T af2f58e6757a4b738a8ec38b2e228f9f P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:00.798105 11040 raft_consensus.cc:374] T af2f58e6757a4b738a8ec38b2e228f9f P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:00.798329 11040 raft_consensus.cc:729] T af2f58e6757a4b738a8ec38b2e228f9f P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:00.802722 10369 catalog_manager.cc:5495] T e0074b1184ec4883ab1016330d98c968 P c8626e300d0644b0908797ad61bd7cef reported cstate change: term changed from 0 to 1, leader changed from <none> to c8626e300d0644b0908797ad61bd7cef (127.10.4.67). New cstate: current_term: 1 leader_uuid: "c8626e300d0644b0908797ad61bd7cef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } health_report { overall_health: UNKNOWN } } }
I20260924 00:21:00.806293 11040 consensus_queue.cc:260] T af2f58e6757a4b738a8ec38b2e228f9f P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:00.807839 10369 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet e0074b1184ec4883ab1016330d98c968 (table default.Table_63927aa1872e41da917d6b92c8acb310 [id=485c42c942b74abbb32bea36d25bc184]). Expected version 2 got 0
I20260924 00:21:00.809276 11039 consensus_queue.cc:260] T eeb11092b803467e9db732468350e852 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:00.809540 11041 consensus_queue.cc:260] T 86374d72238d42c29542e649a7a0200d P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:00.813057 11040 ts_tablet_manager.cc:1428] T af2f58e6757a4b738a8ec38b2e228f9f P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.021s user 0.007s sys 0.000s
I20260924 00:21:00.819427 10372 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 2861c52b3a1e4e36a45e7f4e5d65ac31
I20260924 00:21:00.821939 11041 ts_tablet_manager.cc:1428] T 86374d72238d42c29542e649a7a0200d P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.036s user 0.005s sys 0.000s
I20260924 00:21:00.822754 11041 tablet_bootstrap.cc:492] T d44932f8f08e4ea7acc05b8ec511f045 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:00.823254 11040 tablet_bootstrap.cc:492] T ae0c9b785195491987dd8fe397d704e0 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:00.832172 11041 tablet_bootstrap.cc:654] T d44932f8f08e4ea7acc05b8ec511f045 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:00.825497 11156 ts_tablet_manager.cc:1905] T c212525586544d8f9d92bc4b8bd8acaa P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:00.835296 11040 tablet_bootstrap.cc:654] T ae0c9b785195491987dd8fe397d704e0 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:00.848425 10372 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_6fd2c4af10ab43ffafa1f24e552df8d3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:00.850299 11039 ts_tablet_manager.cc:1428] T eeb11092b803467e9db732468350e852 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.094s user 0.005s sys 0.000s
W20260924 00:21:00.850431 10372 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6fd2c4af10ab43ffafa1f24e552df8d3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:00.851296 11039 tablet_bootstrap.cc:492] T bcd8a6b92b8e492d96994195fa7f3ae5 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:00.860353 11041 tablet_bootstrap.cc:492] T d44932f8f08e4ea7acc05b8ec511f045 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:00.860733 11041 ts_tablet_manager.cc:1397] T d44932f8f08e4ea7acc05b8ec511f045 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.038s user 0.007s sys 0.000s
I20260924 00:21:00.861766 11051 tablet_replica.cc:330] T 453ce17089ce44ba8da67a0893301c3c P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
W20260924 00:21:00.861593 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet c7e50ac8b7f947b1bfba8cd1cff0ea7e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c7e50ac8b7f947b1bfba8cd1cff0ea7e already in progress: creating tablet
I20260924 00:21:00.862840 11041 raft_consensus.cc:348] T d44932f8f08e4ea7acc05b8ec511f045 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:00.863466 11041 raft_consensus.cc:374] T d44932f8f08e4ea7acc05b8ec511f045 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:00.863685 11041 raft_consensus.cc:729] T d44932f8f08e4ea7acc05b8ec511f045 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:00.871023 11051 raft_consensus.cc:2229] T 453ce17089ce44ba8da67a0893301c3c P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:00.873574 11041 consensus_queue.cc:260] T d44932f8f08e4ea7acc05b8ec511f045 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:00.879400 10373 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 95a7a1d2147c4d7ea63ca37a8f0ea4e6
I20260924 00:21:00.879869 11051 raft_consensus.cc:2258] T 453ce17089ce44ba8da67a0893301c3c P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:00.880649 10373 catalog_manager.cc:6565] ReplaceTablet: tablet 95a7a1d2147c4d7ea63ca37a8f0ea4e6 deleted and replaced by tablet 3b8d74761b0b4fd0a22021d3870ec05c
I20260924 00:21:00.881297 10921 tablet_service.cc:1514] Processing DeleteTablet for tablet 95a7a1d2147c4d7ea63ca37a8f0ea4e6 with delete_type TABLET_DATA_DELETED (replaced by tablet 3b8d74761b0b4fd0a22021d3870ec05c) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:00.882066 11041 ts_tablet_manager.cc:1428] T d44932f8f08e4ea7acc05b8ec511f045 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.021s user 0.004s sys 0.000s
I20260924 00:21:00.882099 10776 tablet_service.cc:1514] Processing DeleteTablet for tablet 95a7a1d2147c4d7ea63ca37a8f0ea4e6 with delete_type TABLET_DATA_DELETED (replaced by tablet 3b8d74761b0b4fd0a22021d3870ec05c) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:00.882882 11041 tablet_bootstrap.cc:492] T 85a0ad61b7cb4e11b30e84ddc4655b64 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:00.885121 10632 tablet_service.cc:1514] Processing DeleteTablet for tablet 95a7a1d2147c4d7ea63ca37a8f0ea4e6 with delete_type TABLET_DATA_DELETED (replaced by tablet 3b8d74761b0b4fd0a22021d3870ec05c) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:00.887133 11051 ts_tablet_manager.cc:1905] T 453ce17089ce44ba8da67a0893301c3c P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:00.887573 11039 tablet_bootstrap.cc:654] T bcd8a6b92b8e492d96994195fa7f3ae5 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:00.893460 11041 tablet_bootstrap.cc:654] T 85a0ad61b7cb4e11b30e84ddc4655b64 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:00.898782 11040 tablet_bootstrap.cc:492] T ae0c9b785195491987dd8fe397d704e0 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:00.899663 11040 ts_tablet_manager.cc:1397] T ae0c9b785195491987dd8fe397d704e0 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.077s user 0.011s sys 0.007s
I20260924 00:21:00.902343 11040 raft_consensus.cc:348] T ae0c9b785195491987dd8fe397d704e0 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:00.903064 11040 raft_consensus.cc:374] T ae0c9b785195491987dd8fe397d704e0 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:00.903294 11040 raft_consensus.cc:729] T ae0c9b785195491987dd8fe397d704e0 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:00.903823 11040 consensus_queue.cc:260] T ae0c9b785195491987dd8fe397d704e0 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:00.908084 11156 ts_tablet_manager.cc:1918] T c212525586544d8f9d92bc4b8bd8acaa P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:00.928486 11041 tablet_bootstrap.cc:492] T 85a0ad61b7cb4e11b30e84ddc4655b64 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:00.929049 11041 ts_tablet_manager.cc:1397] T 85a0ad61b7cb4e11b30e84ddc4655b64 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.046s user 0.010s sys 0.002s
I20260924 00:21:00.931202 10365 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 5e624be739394e6d87d6b3fef2f46302
I20260924 00:21:00.931098 11041 raft_consensus.cc:348] T 85a0ad61b7cb4e11b30e84ddc4655b64 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:00.931701 11041 raft_consensus.cc:374] T 85a0ad61b7cb4e11b30e84ddc4655b64 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:00.931939 11041 raft_consensus.cc:729] T 85a0ad61b7cb4e11b30e84ddc4655b64 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:00.932735 10365 catalog_manager.cc:6565] ReplaceTablet: tablet 5e624be739394e6d87d6b3fef2f46302 deleted and replaced by tablet 6bcfd729e9bd4040b3384ab24164e80d
I20260924 00:21:00.933633 10632 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e624be739394e6d87d6b3fef2f46302 with delete_type TABLET_DATA_DELETED (replaced by tablet 6bcfd729e9bd4040b3384ab24164e80d) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:00.934175 10776 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e624be739394e6d87d6b3fef2f46302 with delete_type TABLET_DATA_DELETED (replaced by tablet 6bcfd729e9bd4040b3384ab24164e80d) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:00.934190 10921 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e624be739394e6d87d6b3fef2f46302 with delete_type TABLET_DATA_DELETED (replaced by tablet 6bcfd729e9bd4040b3384ab24164e80d) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:00.929118 11156 log.cc:1196] T c212525586544d8f9d92bc4b8bd8acaa P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/c212525586544d8f9d92bc4b8bd8acaa
I20260924 00:21:00.932607 11041 consensus_queue.cc:260] T 85a0ad61b7cb4e11b30e84ddc4655b64 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:00.935232 11156 ts_tablet_manager.cc:1939] T c212525586544d8f9d92bc4b8bd8acaa P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:00.922281 11040 ts_tablet_manager.cc:1428] T ae0c9b785195491987dd8fe397d704e0 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.022s user 0.005s sys 0.000s
I20260924 00:21:00.937894 11040 tablet_bootstrap.cc:492] T eeb11092b803467e9db732468350e852 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:00.942243 11041 ts_tablet_manager.cc:1428] T 85a0ad61b7cb4e11b30e84ddc4655b64 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.013s user 0.004s sys 0.001s
I20260924 00:21:00.946471 11040 tablet_bootstrap.cc:654] T eeb11092b803467e9db732468350e852 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:00.945617 11041 tablet_bootstrap.cc:492] T 96960f3061fb43e9939b470f49edf1e9 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:00.955518 10365 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_faaa8b08d77d4384a41b2008c9cc98ea"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:00.957192 10365 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_faaa8b08d77d4384a41b2008c9cc98ea in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:00.957363 11049 ts_tablet_manager.cc:1918] T bcd8a6b92b8e492d96994195fa7f3ae5 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:00.957692 11049 log.cc:1196] T bcd8a6b92b8e492d96994195fa7f3ae5 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/bcd8a6b92b8e492d96994195fa7f3ae5
I20260924 00:21:00.958247 11049 ts_tablet_manager.cc:1939] T bcd8a6b92b8e492d96994195fa7f3ae5 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:00.960057 11041 tablet_bootstrap.cc:654] T 96960f3061fb43e9939b470f49edf1e9 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:00.960464 11039 tablet_bootstrap.cc:492] T bcd8a6b92b8e492d96994195fa7f3ae5 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:00.960846 11039 ts_tablet_manager.cc:1397] T bcd8a6b92b8e492d96994195fa7f3ae5 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.110s user 0.009s sys 0.005s
I20260924 00:21:00.962951 11039 raft_consensus.cc:348] T bcd8a6b92b8e492d96994195fa7f3ae5 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:00.963580 11039 raft_consensus.cc:374] T bcd8a6b92b8e492d96994195fa7f3ae5 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:00.963810 11039 raft_consensus.cc:729] T bcd8a6b92b8e492d96994195fa7f3ae5 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:00.964389 11039 consensus_queue.cc:260] T bcd8a6b92b8e492d96994195fa7f3ae5 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:00.968317 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet c212525586544d8f9d92bc4b8bd8acaa (table default.Table_ef16ff2bfc9b42219f0b6d5f957312f6 [id=fbdec8eccace4eb3864a6cdd95795ce3]) successfully deleted
I20260924 00:21:00.979328 10373 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_0adf8e54351d4721846a55224f2831d5" } new_table_name: "default.Table_153086de28a74809b8415724f702a78e" modify_external_catalogs: true
I20260924 00:21:00.979691 11039 ts_tablet_manager.cc:1428] T bcd8a6b92b8e492d96994195fa7f3ae5 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.019s user 0.004s sys 0.000s
I20260924 00:21:00.981779 11051 ts_tablet_manager.cc:1918] T 453ce17089ce44ba8da67a0893301c3c P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:00.982141 11051 log.cc:1196] T 453ce17089ce44ba8da67a0893301c3c P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/453ce17089ce44ba8da67a0893301c3c
I20260924 00:21:00.982725 11051 ts_tablet_manager.cc:1939] T 453ce17089ce44ba8da67a0893301c3c P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:00.985008 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 453ce17089ce44ba8da67a0893301c3c (table default.Table_ca083375f0df4255b8654ed88040dfcd [id=f79ca8d11aa04d5f8f72d58b798ed4ea]) successfully deleted
W20260924 00:21:00.986240 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet ffa011cf8dc446de849bea75a4f20056 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ffa011cf8dc446de849bea75a4f20056 already in progress: creating tablet
I20260924 00:21:00.987224 10366 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet ae0c9b785195491987dd8fe397d704e0
W20260924 00:21:00.989395 10368 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 60ebab3e93f54383b87aeb54bbde722d has no consensus state
I20260924 00:21:00.992800 10632 tablet_service.cc:1514] Processing DeleteTablet for tablet ae0c9b785195491987dd8fe397d704e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:00 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:00.995754 10921 tablet_service.cc:1514] Processing DeleteTablet for tablet ae0c9b785195491987dd8fe397d704e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:00 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:00.966714 11156 tablet_replica.cc:330] T 453ce17089ce44ba8da67a0893301c3c P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:00.988931 10776 tablet_service.cc:1514] Processing DeleteTablet for tablet ae0c9b785195491987dd8fe397d704e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:00 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:01.000464 11039 tablet_bootstrap.cc:492] T 1ec36d59b5114feea144bcb72794136c P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:00.999541 11041 tablet_bootstrap.cc:492] T 96960f3061fb43e9939b470f49edf1e9 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:01.004292 11041 ts_tablet_manager.cc:1397] T 96960f3061fb43e9939b470f49edf1e9 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.059s user 0.019s sys 0.000s
I20260924 00:21:01.004439 11156 raft_consensus.cc:2229] T 453ce17089ce44ba8da67a0893301c3c P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:21:01.005097 11156 pending_rounds.cc:62] T 453ce17089ce44ba8da67a0893301c3c P 48fbd603abe146c585d15196e1b5be08: Trying to abort 1 pending ops.
I20260924 00:21:01.005308 11156 pending_rounds.cc:68] T 453ce17089ce44ba8da67a0893301c3c P 48fbd603abe146c585d15196e1b5be08: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332697130850349056 op_type: NO_OP noop_request { }
I20260924 00:21:01.005648 11156 raft_consensus.cc:2866] T 453ce17089ce44ba8da67a0893301c3c P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260924 00:21:01.006114 11156 raft_consensus.cc:2258] T 453ce17089ce44ba8da67a0893301c3c P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 00:21:01.007220 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 6ac9c7da7fb546f6ae935371799cbad6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6ac9c7da7fb546f6ae935371799cbad6 already in progress: creating tablet
I20260924 00:21:01.007251 11041 raft_consensus.cc:348] T 96960f3061fb43e9939b470f49edf1e9 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:01.008314 11156 ts_tablet_manager.cc:1905] T 453ce17089ce44ba8da67a0893301c3c P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:01.008831 11039 tablet_bootstrap.cc:654] T 1ec36d59b5114feea144bcb72794136c P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:01.009832 11051 tablet_replica.cc:330] T c212525586544d8f9d92bc4b8bd8acaa P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:01.010746 11051 raft_consensus.cc:2229] T c212525586544d8f9d92bc4b8bd8acaa P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:01.011209 11051 raft_consensus.cc:2258] T c212525586544d8f9d92bc4b8bd8acaa P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:01.011835 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet bcd8a6b92b8e492d96994195fa7f3ae5 (table default.Table_99da18207fc542fbb68972724c4d410e [id=f75c1da3bd734e92bc105c473236fa9b]) successfully deleted
I20260924 00:21:01.007875 11041 raft_consensus.cc:374] T 96960f3061fb43e9939b470f49edf1e9 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:01.012706 11051 ts_tablet_manager.cc:1905] T c212525586544d8f9d92bc4b8bd8acaa P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:01.012893 11041 raft_consensus.cc:729] T 96960f3061fb43e9939b470f49edf1e9 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:01.013691 11041 consensus_queue.cc:260] T 96960f3061fb43e9939b470f49edf1e9 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:01.018519 11041 ts_tablet_manager.cc:1428] T 96960f3061fb43e9939b470f49edf1e9 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.014s user 0.005s sys 0.000s
I20260924 00:21:01.023108 11040 tablet_bootstrap.cc:492] T eeb11092b803467e9db732468350e852 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:01.023512 11040 ts_tablet_manager.cc:1397] T eeb11092b803467e9db732468350e852 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.086s user 0.008s sys 0.000s
I20260924 00:21:01.027766 10368 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_9f40e044dcc74a8d90f5fa80c9ec7a26" } modify_external_catalogs: true
I20260924 00:21:01.028259 10368 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_9f40e044dcc74a8d90f5fa80c9ec7a26" } modify_external_catalogs: true
I20260924 00:21:01.012722 11049 tablet_replica.cc:330] T c83942a74816465fa7f3f2c3e18c98bb P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:01.035648 11041 tablet_bootstrap.cc:492] T 7463fcb18e884b1dbc478ebaa19f6cb4 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:01.035944 11040 raft_consensus.cc:348] T eeb11092b803467e9db732468350e852 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:01.036564 11040 raft_consensus.cc:374] T eeb11092b803467e9db732468350e852 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:01.038810 11049 raft_consensus.cc:2229] T c83942a74816465fa7f3f2c3e18c98bb P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:01.039391 11049 raft_consensus.cc:2258] T c83942a74816465fa7f3f2c3e18c98bb P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:01.036794 11040 raft_consensus.cc:729] T eeb11092b803467e9db732468350e852 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:01.043170 11039 tablet_bootstrap.cc:492] T 1ec36d59b5114feea144bcb72794136c P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:01.043552 11039 ts_tablet_manager.cc:1397] T 1ec36d59b5114feea144bcb72794136c P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.043s user 0.009s sys 0.000s
I20260924 00:21:01.043817 11198 raft_consensus.cc:482] T d44932f8f08e4ea7acc05b8ec511f045 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:01.044202 11198 raft_consensus.cc:504] T d44932f8f08e4ea7acc05b8ec511f045 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:01.045446 11040 consensus_queue.cc:260] T eeb11092b803467e9db732468350e852 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:01.045986 11198 leader_election.cc:288] T d44932f8f08e4ea7acc05b8ec511f045 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:01.047366 10941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d44932f8f08e4ea7acc05b8ec511f045" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
W20260924 00:21:01.048576 10589 leader_election.cc:341] T d44932f8f08e4ea7acc05b8ec511f045 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:01.050707 10800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d44932f8f08e4ea7acc05b8ec511f045" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:01.051986 10592 leader_election.cc:341] T d44932f8f08e4ea7acc05b8ec511f045 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:01.051822 11039 raft_consensus.cc:348] T 1ec36d59b5114feea144bcb72794136c P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:01.052443 11039 raft_consensus.cc:374] T 1ec36d59b5114feea144bcb72794136c P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:01.052320 10592 leader_election.cc:302] T d44932f8f08e4ea7acc05b8ec511f045 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:01.052654 11039 raft_consensus.cc:729] T 1ec36d59b5114feea144bcb72794136c P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:01.053315 11198 raft_consensus.cc:2726] T d44932f8f08e4ea7acc05b8ec511f045 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:01.040726 11041 tablet_bootstrap.cc:654] T 7463fcb18e884b1dbc478ebaa19f6cb4 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:01.053221 11039 consensus_queue.cc:260] T 1ec36d59b5114feea144bcb72794136c P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:01.059973 11040 ts_tablet_manager.cc:1428] T eeb11092b803467e9db732468350e852 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.036s user 0.006s sys 0.000s
I20260924 00:21:01.063699 11156 ts_tablet_manager.cc:1918] T 453ce17089ce44ba8da67a0893301c3c P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:01.064028 11156 log.cc:1196] T 453ce17089ce44ba8da67a0893301c3c P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/453ce17089ce44ba8da67a0893301c3c
I20260924 00:21:01.064685 11156 ts_tablet_manager.cc:1939] T 453ce17089ce44ba8da67a0893301c3c P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:01.067850 11049 ts_tablet_manager.cc:1905] T c83942a74816465fa7f3f2c3e18c98bb P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:01.067445 11039 ts_tablet_manager.cc:1428] T 1ec36d59b5114feea144bcb72794136c P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.024s user 0.005s sys 0.001s
I20260924 00:21:01.070174 11041 tablet_bootstrap.cc:492] T 7463fcb18e884b1dbc478ebaa19f6cb4 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:01.070889 11041 ts_tablet_manager.cc:1397] T 7463fcb18e884b1dbc478ebaa19f6cb4 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.035s user 0.003s sys 0.010s
I20260924 00:21:01.072899 11041 raft_consensus.cc:348] T 7463fcb18e884b1dbc478ebaa19f6cb4 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:01.073586 11041 raft_consensus.cc:374] T 7463fcb18e884b1dbc478ebaa19f6cb4 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:01.073822 11041 raft_consensus.cc:729] T 7463fcb18e884b1dbc478ebaa19f6cb4 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:01.074436 11041 consensus_queue.cc:260] T 7463fcb18e884b1dbc478ebaa19f6cb4 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:01.076792 11040 tablet_bootstrap.cc:492] T 1ec36d59b5114feea144bcb72794136c P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:01.078279 11039 tablet_bootstrap.cc:492] T f294831873d548e4b9201c1bbb964096 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:01.081822 11040 tablet_bootstrap.cc:654] T 1ec36d59b5114feea144bcb72794136c P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:01.083242 11041 ts_tablet_manager.cc:1428] T 7463fcb18e884b1dbc478ebaa19f6cb4 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.012s user 0.004s sys 0.000s
I20260924 00:21:01.084713 11051 ts_tablet_manager.cc:1918] T c212525586544d8f9d92bc4b8bd8acaa P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:01.085083 11051 log.cc:1196] T c212525586544d8f9d92bc4b8bd8acaa P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/c212525586544d8f9d92bc4b8bd8acaa
I20260924 00:21:01.085624 11051 ts_tablet_manager.cc:1939] T c212525586544d8f9d92bc4b8bd8acaa P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:01.086736 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 453ce17089ce44ba8da67a0893301c3c (table default.Table_ca083375f0df4255b8654ed88040dfcd [id=f79ca8d11aa04d5f8f72d58b798ed4ea]) successfully deleted
I20260924 00:21:01.087617 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet c212525586544d8f9d92bc4b8bd8acaa (table default.Table_ef16ff2bfc9b42219f0b6d5f957312f6 [id=fbdec8eccace4eb3864a6cdd95795ce3]) successfully deleted
I20260924 00:21:01.084025 11041 tablet_bootstrap.cc:492] T 5b0345252c3c496db2987c30adce3da7 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
W20260924 00:21:01.095207 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 95a7a1d2147c4d7ea63ca37a8f0ea4e6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 95a7a1d2147c4d7ea63ca37a8f0ea4e6 already in progress: creating tablet
I20260924 00:21:01.096268 11199 raft_consensus.cc:482] T eeb11092b803467e9db732468350e852 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:01.096716 11199 raft_consensus.cc:504] T eeb11092b803467e9db732468350e852 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:01.098558 11199 leader_election.cc:288] T eeb11092b803467e9db732468350e852 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
W20260924 00:21:01.102174 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 5e624be739394e6d87d6b3fef2f46302 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5e624be739394e6d87d6b3fef2f46302 already in progress: creating tablet
I20260924 00:21:01.103999 11156 tablet_replica.cc:330] T ae0c9b785195491987dd8fe397d704e0 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:01.105283 11156 raft_consensus.cc:2229] T ae0c9b785195491987dd8fe397d704e0 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:01.105320 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eeb11092b803467e9db732468350e852" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:01.105903 10654 raft_consensus.cc:2454] T eeb11092b803467e9db732468350e852 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 0.
I20260924 00:21:01.107115 11041 tablet_bootstrap.cc:654] T 5b0345252c3c496db2987c30adce3da7 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:01.107163 10776 tablet_service.cc:1514] Processing DeleteTablet for tablet af2f58e6757a4b738a8ec38b2e228f9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:57 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:01.109365 10941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eeb11092b803467e9db732468350e852" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:01.107245 10733 leader_election.cc:302] T eeb11092b803467e9db732468350e852 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670; no voters:
I20260924 00:21:01.109975 10941 raft_consensus.cc:2454] T eeb11092b803467e9db732468350e852 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 0.
I20260924 00:21:01.110360 11199 raft_consensus.cc:2781] T eeb11092b803467e9db732468350e852 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:01.110651 11199 raft_consensus.cc:482] T eeb11092b803467e9db732468350e852 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:01.110946 11199 raft_consensus.cc:3037] T eeb11092b803467e9db732468350e852 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:01.117923 11199 raft_consensus.cc:504] T eeb11092b803467e9db732468350e852 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:01.118778 10371 master_service.cc:946] ReplaceTablet: received request to replace tablet d44932f8f08e4ea7acc05b8ec511f045 from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:01.120633 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eeb11092b803467e9db732468350e852" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:21:01.121047 11199 leader_election.cc:288] T eeb11092b803467e9db732468350e852 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Requested vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:01.121176 10654 raft_consensus.cc:3037] T eeb11092b803467e9db732468350e852 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:01.121644 11156 raft_consensus.cc:2258] T ae0c9b785195491987dd8fe397d704e0 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:01.121683 10941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eeb11092b803467e9db732468350e852" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef"
I20260924 00:21:01.122269 10941 raft_consensus.cc:3037] T eeb11092b803467e9db732468350e852 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:01.123819 11041 tablet_bootstrap.cc:492] T 5b0345252c3c496db2987c30adce3da7 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:01.124183 11041 ts_tablet_manager.cc:1397] T 5b0345252c3c496db2987c30adce3da7 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.040s user 0.006s sys 0.002s
I20260924 00:21:01.126353 11041 raft_consensus.cc:348] T 5b0345252c3c496db2987c30adce3da7 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:01.127171 11041 raft_consensus.cc:374] T 5b0345252c3c496db2987c30adce3da7 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:01.127350 10654 raft_consensus.cc:2454] T eeb11092b803467e9db732468350e852 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 1.
I20260924 00:21:01.127410 11041 raft_consensus.cc:729] T 5b0345252c3c496db2987c30adce3da7 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:01.128782 10733 leader_election.cc:302] T eeb11092b803467e9db732468350e852 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670; no voters:
I20260924 00:21:01.129580 11199 raft_consensus.cc:2781] T eeb11092b803467e9db732468350e852 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:01.129168 11041 consensus_queue.cc:260] T 5b0345252c3c496db2987c30adce3da7 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:01.131047 11199 raft_consensus.cc:686] T eeb11092b803467e9db732468350e852 P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: Becoming Leader. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Running, Role: LEADER
I20260924 00:21:01.132531 11199 consensus_queue.cc:237] T eeb11092b803467e9db732468350e852 P 48fbd603abe146c585d15196e1b5be08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:01.136008 10941 raft_consensus.cc:2454] T eeb11092b803467e9db732468350e852 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 1.
I20260924 00:21:01.136648 11198 raft_consensus.cc:482] T 86374d72238d42c29542e649a7a0200d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:01.137049 11198 raft_consensus.cc:504] T 86374d72238d42c29542e649a7a0200d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:01.138693 11198 leader_election.cc:288] T 86374d72238d42c29542e649a7a0200d P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:01.140002 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86374d72238d42c29542e649a7a0200d" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:01.140852 11041 ts_tablet_manager.cc:1428] T 5b0345252c3c496db2987c30adce3da7 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.016s user 0.006s sys 0.001s
W20260924 00:21:01.141404 10589 leader_election.cc:341] T 86374d72238d42c29542e649a7a0200d P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:01.141856 11039 tablet_bootstrap.cc:654] T f294831873d548e4b9201c1bbb964096 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:01.143460 11041 tablet_bootstrap.cc:492] T 59c077eb68794340ac900dbedc7056ee P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:01.149312 11051 tablet_replica.cc:330] T af2f58e6757a4b738a8ec38b2e228f9f P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:01.150056 11051 raft_consensus.cc:2229] T af2f58e6757a4b738a8ec38b2e228f9f P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:01.150563 11051 raft_consensus.cc:2258] T af2f58e6757a4b738a8ec38b2e228f9f P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:01.136293 11040 tablet_bootstrap.cc:492] T 1ec36d59b5114feea144bcb72794136c P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:01.140686 10800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86374d72238d42c29542e649a7a0200d" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:01.154446 10592 leader_election.cc:341] T 86374d72238d42c29542e649a7a0200d P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:01.154835 10592 leader_election.cc:302] T 86374d72238d42c29542e649a7a0200d P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:01.155730 11198 raft_consensus.cc:2726] T 86374d72238d42c29542e649a7a0200d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:01.156654 11040 ts_tablet_manager.cc:1397] T 1ec36d59b5114feea144bcb72794136c P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.080s user 0.010s sys 0.000s
I20260924 00:21:01.159175 11040 raft_consensus.cc:348] T 1ec36d59b5114feea144bcb72794136c P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:01.148844 11041 tablet_bootstrap.cc:654] T 59c077eb68794340ac900dbedc7056ee P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:01.159781 11040 raft_consensus.cc:374] T 1ec36d59b5114feea144bcb72794136c P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:01.160012 11040 raft_consensus.cc:729] T 1ec36d59b5114feea144bcb72794136c P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:01.161805 11051 ts_tablet_manager.cc:1905] T af2f58e6757a4b738a8ec38b2e228f9f P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:01.167881 11039 tablet_bootstrap.cc:492] T f294831873d548e4b9201c1bbb964096 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:01.168311 11039 ts_tablet_manager.cc:1397] T f294831873d548e4b9201c1bbb964096 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.090s user 0.006s sys 0.003s
I20260924 00:21:01.152279 11156 ts_tablet_manager.cc:1905] T ae0c9b785195491987dd8fe397d704e0 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:01.160544 11040 consensus_queue.cc:260] T 1ec36d59b5114feea144bcb72794136c P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:01.170845 11039 raft_consensus.cc:348] T f294831873d548e4b9201c1bbb964096 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:01.176568 11039 raft_consensus.cc:374] T f294831873d548e4b9201c1bbb964096 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:01.176843 11039 raft_consensus.cc:729] T f294831873d548e4b9201c1bbb964096 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:01.179131 11039 consensus_queue.cc:260] T f294831873d548e4b9201c1bbb964096 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:01.181110 11041 tablet_bootstrap.cc:492] T 59c077eb68794340ac900dbedc7056ee P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:01.181509 11041 ts_tablet_manager.cc:1397] T 59c077eb68794340ac900dbedc7056ee P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.040s user 0.007s sys 0.004s
I20260924 00:21:01.183507 11041 raft_consensus.cc:348] T 59c077eb68794340ac900dbedc7056ee P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:01.189363 11040 ts_tablet_manager.cc:1428] T 1ec36d59b5114feea144bcb72794136c P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.032s user 0.005s sys 0.000s
I20260924 00:21:01.189684 11041 raft_consensus.cc:374] T 59c077eb68794340ac900dbedc7056ee P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:01.195319 11041 raft_consensus.cc:729] T 59c077eb68794340ac900dbedc7056ee P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:01.197707 11039 ts_tablet_manager.cc:1428] T f294831873d548e4b9201c1bbb964096 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.029s user 0.003s sys 0.003s
I20260924 00:21:01.195968 11041 consensus_queue.cc:260] T 59c077eb68794340ac900dbedc7056ee P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:01.201146 11040 tablet_bootstrap.cc:492] T bcd8a6b92b8e492d96994195fa7f3ae5 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:01.203783 11041 ts_tablet_manager.cc:1428] T 59c077eb68794340ac900dbedc7056ee P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.022s user 0.003s sys 0.001s
I20260924 00:21:01.203948 10365 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_236b1998bb2f474c9e5452abe4c83c64" } new_table_name: "default.Table_fd43d6016c81497586729ff4cbe3213c" modify_external_catalogs: true
I20260924 00:21:01.207118 11039 tablet_bootstrap.cc:492] T 85a0ad61b7cb4e11b30e84ddc4655b64 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:01.208407 10372 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_bdbcdcde9ac241d5b22b5e8b0da6a9a7" } modify_external_catalogs: true
I20260924 00:21:01.209097 10372 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_bdbcdcde9ac241d5b22b5e8b0da6a9a7" } modify_external_catalogs: true
I20260924 00:21:01.205452 11198 raft_consensus.cc:482] T 5d7d52be4f1f4fc29b8172b9fa878f76 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:01.213406 11198 raft_consensus.cc:504] T 5d7d52be4f1f4fc29b8172b9fa878f76 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:01.213963 11040 tablet_bootstrap.cc:654] T bcd8a6b92b8e492d96994195fa7f3ae5 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:01.215233 11198 leader_election.cc:288] T 5d7d52be4f1f4fc29b8172b9fa878f76 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:01.216681 10941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d7d52be4f1f4fc29b8172b9fa878f76" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:01.217306 11041 tablet_bootstrap.cc:492] T fcf651adb1d3459facfe3a8df2a91ed7 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:01.217656 10800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d7d52be4f1f4fc29b8172b9fa878f76" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:01.218163 10589 leader_election.cc:341] T 5d7d52be4f1f4fc29b8172b9fa878f76 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
W20260924 00:21:01.220000 10592 leader_election.cc:341] T 5d7d52be4f1f4fc29b8172b9fa878f76 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:01.220359 10592 leader_election.cc:302] T 5d7d52be4f1f4fc29b8172b9fa878f76 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:01.220541 11039 tablet_bootstrap.cc:654] T 85a0ad61b7cb4e11b30e84ddc4655b64 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:01.222007 11198 raft_consensus.cc:2726] T 5d7d52be4f1f4fc29b8172b9fa878f76 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:01.224892 11051 ts_tablet_manager.cc:1918] T af2f58e6757a4b738a8ec38b2e228f9f P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:01.225267 11051 log.cc:1196] T af2f58e6757a4b738a8ec38b2e228f9f P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/af2f58e6757a4b738a8ec38b2e228f9f
I20260924 00:21:01.225870 11051 ts_tablet_manager.cc:1939] T af2f58e6757a4b738a8ec38b2e228f9f P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:01.229396 11040 tablet_bootstrap.cc:492] T bcd8a6b92b8e492d96994195fa7f3ae5 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:01.229858 11040 ts_tablet_manager.cc:1397] T bcd8a6b92b8e492d96994195fa7f3ae5 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.029s user 0.008s sys 0.000s
W20260924 00:21:01.230777 10373 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0980afc2af174da2a8ca2dd414e9825c consensus state has no leader
I20260924 00:21:01.234014 11198 raft_consensus.cc:482] T 59c077eb68794340ac900dbedc7056ee P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:01.234453 11198 raft_consensus.cc:504] T 59c077eb68794340ac900dbedc7056ee P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:01.236980 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet af2f58e6757a4b738a8ec38b2e228f9f (table default.Table_391ed8efc7a54515837c1b79be85d64c [id=f72ddb5979044cda93de75fcb1983ca4]) successfully deleted
I20260924 00:21:01.237331 10941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59c077eb68794340ac900dbedc7056ee" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:01.237634 11051 tablet_replica.cc:330] T ae0c9b785195491987dd8fe397d704e0 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:01.237710 11198 leader_election.cc:288] T 59c077eb68794340ac900dbedc7056ee P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
W20260924 00:21:01.239035 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 95a7a1d2147c4d7ea63ca37a8f0ea4e6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 95a7a1d2147c4d7ea63ca37a8f0ea4e6 already in progress: creating tablet
W20260924 00:21:01.239097 10589 leader_election.cc:341] T 59c077eb68794340ac900dbedc7056ee P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
W20260924 00:21:01.239643 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 5e624be739394e6d87d6b3fef2f46302 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5e624be739394e6d87d6b3fef2f46302 already in progress: creating tablet
I20260924 00:21:01.239904 11040 raft_consensus.cc:348] T bcd8a6b92b8e492d96994195fa7f3ae5 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:01.240543 11040 raft_consensus.cc:374] T bcd8a6b92b8e492d96994195fa7f3ae5 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:01.240403 10800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59c077eb68794340ac900dbedc7056ee" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:01.240775 11040 raft_consensus.cc:729] T bcd8a6b92b8e492d96994195fa7f3ae5 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:01.243700 11040 consensus_queue.cc:260] T bcd8a6b92b8e492d96994195fa7f3ae5 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:01.245702 11156 ts_tablet_manager.cc:1918] T ae0c9b785195491987dd8fe397d704e0 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:01.246032 11156 log.cc:1196] T ae0c9b785195491987dd8fe397d704e0 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/ae0c9b785195491987dd8fe397d704e0
I20260924 00:21:01.246613 11156 ts_tablet_manager.cc:1939] T ae0c9b785195491987dd8fe397d704e0 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:01.246627 11039 tablet_bootstrap.cc:492] T 85a0ad61b7cb4e11b30e84ddc4655b64 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:01.246996 11039 ts_tablet_manager.cc:1397] T 85a0ad61b7cb4e11b30e84ddc4655b64 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.040s user 0.005s sys 0.003s
W20260924 00:21:01.248778 10592 leader_election.cc:341] T 59c077eb68794340ac900dbedc7056ee P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:01.249214 10592 leader_election.cc:302] T 59c077eb68794340ac900dbedc7056ee P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:01.250372 10373 master_service.cc:946] ReplaceTablet: received request to replace tablet 2964cd68d5e94f3780a6a3157bd86983 from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:01.251132 11040 ts_tablet_manager.cc:1428] T bcd8a6b92b8e492d96994195fa7f3ae5 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.021s user 0.005s sys 0.000s
I20260924 00:21:01.238270 11051 raft_consensus.cc:2229] T ae0c9b785195491987dd8fe397d704e0 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:01.253194 11051 raft_consensus.cc:2258] T ae0c9b785195491987dd8fe397d704e0 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:01.253067 11198 raft_consensus.cc:2726] T 59c077eb68794340ac900dbedc7056ee P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:01.253574 11039 raft_consensus.cc:348] T 85a0ad61b7cb4e11b30e84ddc4655b64 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:01.254230 11039 raft_consensus.cc:374] T 85a0ad61b7cb4e11b30e84ddc4655b64 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:01.254453 11039 raft_consensus.cc:729] T 85a0ad61b7cb4e11b30e84ddc4655b64 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:01.255988 11051 ts_tablet_manager.cc:1905] T ae0c9b785195491987dd8fe397d704e0 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:01.258322 11041 tablet_bootstrap.cc:654] T fcf651adb1d3459facfe3a8df2a91ed7 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:01.258419 10373 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_7b8c1651b4974bd5b74177dddadf5c12"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:01.258898 11039 consensus_queue.cc:260] T 85a0ad61b7cb4e11b30e84ddc4655b64 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
W20260924 00:21:01.260156 10373 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7b8c1651b4974bd5b74177dddadf5c12 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:01.264837 11049 ts_tablet_manager.cc:1918] T c83942a74816465fa7f3f2c3e18c98bb P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:01.265175 11049 log.cc:1196] T c83942a74816465fa7f3f2c3e18c98bb P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/c83942a74816465fa7f3f2c3e18c98bb
I20260924 00:21:01.265736 11049 ts_tablet_manager.cc:1939] T c83942a74816465fa7f3f2c3e18c98bb P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:01.266830 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet ae0c9b785195491987dd8fe397d704e0 (table default.Table_d61af65b53ce4871b7b24752281af745 [id=0edc6c93a1ba4c8d81755129301c83de]) successfully deleted
I20260924 00:21:01.267629 11156 tablet_replica.cc:330] T af2f58e6757a4b738a8ec38b2e228f9f P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:01.269882 11039 ts_tablet_manager.cc:1428] T 85a0ad61b7cb4e11b30e84ddc4655b64 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.023s user 0.005s sys 0.001s
I20260924 00:21:01.271246 11040 tablet_bootstrap.cc:492] T 96960f3061fb43e9939b470f49edf1e9 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:01.271565 11156 raft_consensus.cc:2229] T af2f58e6757a4b738a8ec38b2e228f9f P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:01.271986 11156 raft_consensus.cc:2258] T af2f58e6757a4b738a8ec38b2e228f9f P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:01.272790 10371 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_29e523da31634984b775f31084e82c99"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:01.274197 10371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_29e523da31634984b775f31084e82c99 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:01.277704 11198 raft_consensus.cc:482] T 7463fcb18e884b1dbc478ebaa19f6cb4 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:01.278111 11198 raft_consensus.cc:504] T 7463fcb18e884b1dbc478ebaa19f6cb4 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:01.280681 10941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7463fcb18e884b1dbc478ebaa19f6cb4" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
W20260924 00:21:01.282014 10589 leader_election.cc:341] T 7463fcb18e884b1dbc478ebaa19f6cb4 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:01.282502 11156 ts_tablet_manager.cc:1905] T af2f58e6757a4b738a8ec38b2e228f9f P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:01.285181 11198 leader_election.cc:288] T 7463fcb18e884b1dbc478ebaa19f6cb4 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:21:01.285424 10800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7463fcb18e884b1dbc478ebaa19f6cb4" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:01.289897 11039 tablet_bootstrap.cc:492] T 96960f3061fb43e9939b470f49edf1e9 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:01.289965 11040 tablet_bootstrap.cc:654] T 96960f3061fb43e9939b470f49edf1e9 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:01.300105 10368 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 3172bf5793a14655a406ce4d1f086e95
W20260924 00:21:01.286630 10592 leader_election.cc:341] T 7463fcb18e884b1dbc478ebaa19f6cb4 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:01.301069 10592 leader_election.cc:302] T 7463fcb18e884b1dbc478ebaa19f6cb4 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:01.290529 11041 tablet_bootstrap.cc:492] T fcf651adb1d3459facfe3a8df2a91ed7 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:01.302006 11039 tablet_bootstrap.cc:654] T 96960f3061fb43e9939b470f49edf1e9 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:01.302424 11041 ts_tablet_manager.cc:1397] T fcf651adb1d3459facfe3a8df2a91ed7 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.085s user 0.012s sys 0.000s
I20260924 00:21:01.303092 11040 tablet_bootstrap.cc:492] T 96960f3061fb43e9939b470f49edf1e9 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:01.303534 11040 ts_tablet_manager.cc:1397] T 96960f3061fb43e9939b470f49edf1e9 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.032s user 0.008s sys 0.000s
I20260924 00:21:01.303918 11198 raft_consensus.cc:2726] T 7463fcb18e884b1dbc478ebaa19f6cb4 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:01.305941 11041 raft_consensus.cc:348] T fcf651adb1d3459facfe3a8df2a91ed7 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:01.306597 11041 raft_consensus.cc:374] T fcf651adb1d3459facfe3a8df2a91ed7 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:01.306885 11041 raft_consensus.cc:729] T fcf651adb1d3459facfe3a8df2a91ed7 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:01.307526 11041 consensus_queue.cc:260] T fcf651adb1d3459facfe3a8df2a91ed7 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:01.315026 11051 ts_tablet_manager.cc:1918] T ae0c9b785195491987dd8fe397d704e0 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:01.315436 11051 log.cc:1196] T ae0c9b785195491987dd8fe397d704e0 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/ae0c9b785195491987dd8fe397d704e0
I20260924 00:21:01.316051 11051 ts_tablet_manager.cc:1939] T ae0c9b785195491987dd8fe397d704e0 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:01.317292 11041 ts_tablet_manager.cc:1428] T fcf651adb1d3459facfe3a8df2a91ed7 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.015s user 0.005s sys 0.000s
I20260924 00:21:01.317319 11040 raft_consensus.cc:348] T 96960f3061fb43e9939b470f49edf1e9 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:01.317962 11040 raft_consensus.cc:374] T 96960f3061fb43e9939b470f49edf1e9 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:01.318203 11040 raft_consensus.cc:729] T 96960f3061fb43e9939b470f49edf1e9 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:01.320272 11040 consensus_queue.cc:260] T 96960f3061fb43e9939b470f49edf1e9 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:01.323575 10370 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 1ec36d59b5114feea144bcb72794136c (table default.Table_b1726c09065e4fbaa2e6799b3ebdbc74 [id=4dbb68e0a32349ad8da2945db650584d]). Expected version 1 got 0
I20260924 00:21:01.325399 10921 tablet_service.cc:1467] Processing CreateTablet for tablet 8a9443159a334093b4135364057e0c8b (DEFAULT_TABLE table=default.Table_08bc69c89b564e158a0b6d764972cb19 [id=68d73a1a9f1540809f974b9a21f44c1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:01.323565 10632 tablet_service.cc:1467] Processing CreateTablet for tablet 8a9443159a334093b4135364057e0c8b (DEFAULT_TABLE table=default.Table_08bc69c89b564e158a0b6d764972cb19 [id=68d73a1a9f1540809f974b9a21f44c1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:01.326460 10921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a9443159a334093b4135364057e0c8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:01.326905 10632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a9443159a334093b4135364057e0c8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:01.326905 11041 tablet_bootstrap.cc:492] T 666755ec87bb496db75e43ec9204ec5b P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:01.334616 10776 tablet_service.cc:1467] Processing CreateTablet for tablet 8a9443159a334093b4135364057e0c8b (DEFAULT_TABLE table=default.Table_08bc69c89b564e158a0b6d764972cb19 [id=68d73a1a9f1540809f974b9a21f44c1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:01.335681 10776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a9443159a334093b4135364057e0c8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:01.340692 11218 raft_consensus.cc:482] T bcd8a6b92b8e492d96994195fa7f3ae5 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:01.341145 11218 raft_consensus.cc:504] T bcd8a6b92b8e492d96994195fa7f3ae5 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:01.324220 10370 catalog_manager.cc:5495] T ccb26e0fbc1948bc9a00e9e26c4d1139 P b10a9082a37b46f09d581944914c1670 reported cstate change: term changed from 0 to 1, leader changed from <none> to b10a9082a37b46f09d581944914c1670 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "b10a9082a37b46f09d581944914c1670" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } health_report { overall_health: UNKNOWN } } }
I20260924 00:21:01.342979 11218 leader_election.cc:288] T bcd8a6b92b8e492d96994195fa7f3ae5 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:21:01.343320 10370 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet ccb26e0fbc1948bc9a00e9e26c4d1139 (table default.Table_01b5209b9d5c4115b0bf6aef48b51bd8 [id=7e9e663acedf4a1fae2f04cde97b7907]). Expected version 1 got 0
I20260924 00:21:01.348498 10634 tablet_service.cc:1467] Processing CreateTablet for tablet a72742864a6b4299b71757fb308dcadf (DEFAULT_TABLE table=default.Table_fe2e015f4c3d44f5a3ed8797af219464 [id=8264957c289442e0b31b70cb7e17f7bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:01.349669 10634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a72742864a6b4299b71757fb308dcadf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:01.361016 11198 raft_consensus.cc:482] T 85a0ad61b7cb4e11b30e84ddc4655b64 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:01.361429 10921 tablet_service.cc:1467] Processing CreateTablet for tablet a72742864a6b4299b71757fb308dcadf (DEFAULT_TABLE table=default.Table_fe2e015f4c3d44f5a3ed8797af219464 [id=8264957c289442e0b31b70cb7e17f7bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:01.361421 11198 raft_consensus.cc:504] T 85a0ad61b7cb4e11b30e84ddc4655b64 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:01.362367 10921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a72742864a6b4299b71757fb308dcadf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:01.362471 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet ae0c9b785195491987dd8fe397d704e0 (table default.Table_d61af65b53ce4871b7b24752281af745 [id=0edc6c93a1ba4c8d81755129301c83de]) successfully deleted
I20260924 00:21:01.364436 11198 leader_election.cc:288] T 85a0ad61b7cb4e11b30e84ddc4655b64 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:01.364600 10941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85a0ad61b7cb4e11b30e84ddc4655b64" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:01.365164 10941 raft_consensus.cc:2454] T 85a0ad61b7cb4e11b30e84ddc4655b64 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b10a9082a37b46f09d581944914c1670 in term 0.
I20260924 00:21:01.366844 10589 leader_election.cc:302] T 85a0ad61b7cb4e11b30e84ddc4655b64 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:01.367679 11198 raft_consensus.cc:482] T 96960f3061fb43e9939b470f49edf1e9 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:01.368053 11198 raft_consensus.cc:504] T 96960f3061fb43e9939b470f49edf1e9 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:01.373996 11218 raft_consensus.cc:482] T 1ec36d59b5114feea144bcb72794136c P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:01.374388 11218 raft_consensus.cc:504] T 1ec36d59b5114feea144bcb72794136c P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:01.377019 11218 leader_election.cc:288] T 1ec36d59b5114feea144bcb72794136c P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:01.377611 10922 tablet_service.cc:1467] Processing CreateTablet for tablet 0c583d9ba1414f8ea2e605b4b6b076f8 (DEFAULT_TABLE table=default.Table_236b1998bb2f474c9e5452abe4c83c64 [id=70e3fa6998b649468ec8575758708401]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:01.378614 10922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c583d9ba1414f8ea2e605b4b6b076f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:01.384030 11198 leader_election.cc:288] T 96960f3061fb43e9939b470f49edf1e9 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:01.385107 10941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96960f3061fb43e9939b470f49edf1e9" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
W20260924 00:21:01.386327 10589 leader_election.cc:341] T 96960f3061fb43e9939b470f49edf1e9 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:01.388856 11198 raft_consensus.cc:2781] T 85a0ad61b7cb4e11b30e84ddc4655b64 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:01.389106 11198 raft_consensus.cc:482] T 85a0ad61b7cb4e11b30e84ddc4655b64 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:01.389293 11198 raft_consensus.cc:3037] T 85a0ad61b7cb4e11b30e84ddc4655b64 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:01.391585 10921 tablet_service.cc:1467] Processing CreateTablet for tablet 60ebab3e93f54383b87aeb54bbde722d (DEFAULT_TABLE table=default.Table_d952b34e7fe64fe494f56df574e61d77 [id=ed6b4628a295498da00f778ad37db72e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:01.392452 10921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60ebab3e93f54383b87aeb54bbde722d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:01.394526 11041 tablet_bootstrap.cc:654] T 666755ec87bb496db75e43ec9204ec5b P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:01.402554 11039 tablet_bootstrap.cc:492] T 96960f3061fb43e9939b470f49edf1e9 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:01.402683 10915 tablet_service.cc:1467] Processing CreateTablet for tablet a0f71049f6b3478da3b07ebc9293ce64 (DEFAULT_TABLE table=default.Table_9f2df24361da4f9a9340f6f8673043ea [id=59ec3e94e2f44150994a5ffc1d9fa8f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:01.403789 10915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0f71049f6b3478da3b07ebc9293ce64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:01.408983 11039 ts_tablet_manager.cc:1397] T 96960f3061fb43e9939b470f49edf1e9 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.119s user 0.005s sys 0.005s
I20260924 00:21:01.410717 11198 raft_consensus.cc:504] T 85a0ad61b7cb4e11b30e84ddc4655b64 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:01.415299 11252 consensus_queue.cc:1035] T ccb26e0fbc1948bc9a00e9e26c4d1139 P b10a9082a37b46f09d581944914c1670 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:01.417347 10941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85a0ad61b7cb4e11b30e84ddc4655b64" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef"
I20260924 00:21:01.417990 10941 raft_consensus.cc:3037] T 85a0ad61b7cb4e11b30e84ddc4655b64 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:01.427351 10367 master_service.cc:946] ReplaceTablet: received request to replace tablet ccb26e0fbc1948bc9a00e9e26c4d1139 from {username='slave'} at 127.0.0.1:47022
W20260924 00:21:01.432399 10876 leader_election.cc:341] T bcd8a6b92b8e492d96994195fa7f3ae5 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: bcd8a6b92b8e492d96994195fa7f3ae5
I20260924 00:21:01.336670 11156 ts_tablet_manager.cc:1918] T af2f58e6757a4b738a8ec38b2e228f9f P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:01.338043 11199 raft_consensus.cc:482] T 1ec36d59b5114feea144bcb72794136c P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:01.435833 11199 raft_consensus.cc:504] T 1ec36d59b5114feea144bcb72794136c P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:01.437569 11199 leader_election.cc:288] T 1ec36d59b5114feea144bcb72794136c P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:01.339226 10780 tablet_service.cc:1514] Processing DeleteTablet for tablet ae0c9b785195491987dd8fe397d704e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:00 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:01.339655 10775 tablet_service.cc:1514] Processing DeleteTablet for tablet eeb11092b803467e9db732468350e852 with delete_type TABLET_DATA_DELETED (replaced by tablet 4325b60efc1649c9ba4828d0a098a676) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:01.439469 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ec36d59b5114feea144bcb72794136c" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:01.439996 10942 raft_consensus.cc:2454] T 1ec36d59b5114feea144bcb72794136c P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 0.
I20260924 00:21:01.344545 11040 ts_tablet_manager.cc:1428] T 96960f3061fb43e9939b470f49edf1e9 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.041s user 0.008s sys 0.000s
I20260924 00:21:01.440979 11040 tablet_bootstrap.cc:492] T d44932f8f08e4ea7acc05b8ec511f045 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:01.353961 10774 tablet_service.cc:1467] Processing CreateTablet for tablet a72742864a6b4299b71757fb308dcadf (DEFAULT_TABLE table=default.Table_fe2e015f4c3d44f5a3ed8797af219464 [id=8264957c289442e0b31b70cb7e17f7bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:01.365863 10800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85a0ad61b7cb4e11b30e84ddc4655b64" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:01.366640 10799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcd8a6b92b8e492d96994195fa7f3ae5" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:01.443537 10799 raft_consensus.cc:2454] T bcd8a6b92b8e492d96994195fa7f3ae5 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
I20260924 00:21:01.367760 11150 raft_consensus.cc:482] T e0074b1184ec4883ab1016330d98c968 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:01.444020 11150 raft_consensus.cc:504] T e0074b1184ec4883ab1016330d98c968 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:01.444579 10878 leader_election.cc:302] T bcd8a6b92b8e492d96994195fa7f3ae5 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters: b10a9082a37b46f09d581944914c1670
I20260924 00:21:01.445636 11218 raft_consensus.cc:2781] T bcd8a6b92b8e492d96994195fa7f3ae5 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:01.445786 11150 leader_election.cc:288] T e0074b1184ec4883ab1016330d98c968 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:01.369263 10773 tablet_service.cc:1467] Processing CreateTablet for tablet 0c583d9ba1414f8ea2e605b4b6b076f8 (DEFAULT_TABLE table=default.Table_236b1998bb2f474c9e5452abe4c83c64 [id=70e3fa6998b649468ec8575758708401]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:01.370679 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96960f3061fb43e9939b470f49edf1e9" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:01.445955 11218 raft_consensus.cc:482] T bcd8a6b92b8e492d96994195fa7f3ae5 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:01.446254 11218 raft_consensus.cc:3037] T bcd8a6b92b8e492d96994195fa7f3ae5 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:01.446419 10797 raft_consensus.cc:2454] T 96960f3061fb43e9939b470f49edf1e9 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b10a9082a37b46f09d581944914c1670 in term 0.
I20260924 00:21:01.378053 10798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ec36d59b5114feea144bcb72794136c" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:01.447021 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a72742864a6b4299b71757fb308dcadf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:01.447098 10798 raft_consensus.cc:2454] T 1ec36d59b5114feea144bcb72794136c P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
I20260924 00:21:01.385439 10778 tablet_service.cc:1467] Processing CreateTablet for tablet 60ebab3e93f54383b87aeb54bbde722d (DEFAULT_TABLE table=default.Table_d952b34e7fe64fe494f56df574e61d77 [id=ed6b4628a295498da00f778ad37db72e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:01.400763 10777 tablet_service.cc:1467] Processing CreateTablet for tablet a0f71049f6b3478da3b07ebc9293ce64 (DEFAULT_TABLE table=default.Table_9f2df24361da4f9a9340f6f8673043ea [id=59ec3e94e2f44150994a5ffc1d9fa8f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:01.415488 10796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85a0ad61b7cb4e11b30e84ddc4655b64" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:21:01.428706 10779 tablet_service.cc:1467] Processing CreateTablet for tablet 2499754460e04c27ac53d52a4a67f557 (DEFAULT_TABLE table=default.Table_6cf69012192f4f1087e29db70ae8c282 [id=93c1734de58245f8b1d5ab88ec9f0c8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:01.449719 10778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60ebab3e93f54383b87aeb54bbde722d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:01.452354 10732 leader_election.cc:302] T 1ec36d59b5114feea144bcb72794136c P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:01.453444 10878 leader_election.cc:302] T 1ec36d59b5114feea144bcb72794136c P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:01.453693 10779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2499754460e04c27ac53d52a4a67f557. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:01.455206 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0074b1184ec4883ab1016330d98c968" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:01.457149 11150 raft_consensus.cc:2781] T 1ec36d59b5114feea144bcb72794136c P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:01.457492 11150 raft_consensus.cc:482] T 1ec36d59b5114feea144bcb72794136c P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:01.457778 11150 raft_consensus.cc:3037] T 1ec36d59b5114feea144bcb72794136c P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:01.457916 10773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c583d9ba1414f8ea2e605b4b6b076f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:01.471503 10777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0f71049f6b3478da3b07ebc9293ce64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:01.477141 11150 raft_consensus.cc:504] T 1ec36d59b5114feea144bcb72794136c P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:01.478416 11218 raft_consensus.cc:504] T bcd8a6b92b8e492d96994195fa7f3ae5 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:01.480515 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ec36d59b5114feea144bcb72794136c" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef"
I20260924 00:21:01.481501 10942 raft_consensus.cc:3037] T 1ec36d59b5114feea144bcb72794136c P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:01.481909 10921 tablet_service.cc:1467] Processing CreateTablet for tablet 2499754460e04c27ac53d52a4a67f557 (DEFAULT_TABLE table=default.Table_6cf69012192f4f1087e29db70ae8c282 [id=93c1734de58245f8b1d5ab88ec9f0c8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:01.483052 10921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2499754460e04c27ac53d52a4a67f557. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:01.497187 11039 raft_consensus.cc:348] T 96960f3061fb43e9939b470f49edf1e9 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:01.498072 11039 raft_consensus.cc:374] T 96960f3061fb43e9939b470f49edf1e9 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:01.499778 10942 raft_consensus.cc:2454] T 1ec36d59b5114feea144bcb72794136c P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 1.
I20260924 00:21:01.502755 10369 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_22740cf31fbe40a58c31252f63417ad3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:01.504225 10369 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_22740cf31fbe40a58c31252f63417ad3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260924 00:21:01.504727 10361 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet c83942a74816465fa7f3f2c3e18c98bb (table default.Table_41d1376556a9466fa348723fd8e2b2be [id=7eb84ebbe346450eb2b944db685c414b], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet c83942a74816465fa7f3f2c3e18c98bb consensus state has no leader
I20260924 00:21:01.505647 10368 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_b617c841f36c48be8b10898aff4bda2e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:01.507059 10368 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b617c841f36c48be8b10898aff4bda2e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:01.446076 10941 raft_consensus.cc:2454] T 85a0ad61b7cb4e11b30e84ddc4655b64 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b10a9082a37b46f09d581944914c1670 in term 1.
W20260924 00:21:01.512542 10365 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0c583d9ba1414f8ea2e605b4b6b076f8 consensus state has no leader
I20260924 00:21:01.512966 10732 leader_election.cc:302] T 1ec36d59b5114feea144bcb72794136c P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:01.513881 10589 leader_election.cc:302] T 85a0ad61b7cb4e11b30e84ddc4655b64 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:01.514408 10365 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_a6631f3ce9a04312b19aa2e85b4c9fe3" } new_table_name: "default.Table_76ffb857cf034b6f80598edd90dad953" modify_external_catalogs: true
I20260924 00:21:01.516217 11199 raft_consensus.cc:2781] T 1ec36d59b5114feea144bcb72794136c P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:01.517655 11251 raft_consensus.cc:2781] T 85a0ad61b7cb4e11b30e84ddc4655b64 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:01.416448 11049 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 99) took 1304 ms. Trace:
I20260924 00:21:01.421211 10635 tablet_service.cc:1467] Processing CreateTablet for tablet 0c583d9ba1414f8ea2e605b4b6b076f8 (DEFAULT_TABLE table=default.Table_236b1998bb2f474c9e5452abe4c83c64 [id=70e3fa6998b649468ec8575758708401]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:01.422684 10636 tablet_service.cc:1467] Processing CreateTablet for tablet 60ebab3e93f54383b87aeb54bbde722d (DEFAULT_TABLE table=default.Table_d952b34e7fe64fe494f56df574e61d77 [id=ed6b4628a295498da00f778ad37db72e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:01.423035 10630 tablet_service.cc:1467] Processing CreateTablet for tablet a0f71049f6b3478da3b07ebc9293ce64 (DEFAULT_TABLE table=default.Table_9f2df24361da4f9a9340f6f8673043ea [id=59ec3e94e2f44150994a5ffc1d9fa8f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:01.519033 10635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c583d9ba1414f8ea2e605b4b6b076f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:01.519553 10630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0f71049f6b3478da3b07ebc9293ce64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:01.522641 10373 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_1347839140d544c3b4aee063e2efe906" } modify_external_catalogs: true
I20260924 00:21:01.523147 10373 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_1347839140d544c3b4aee063e2efe906" } modify_external_catalogs: true
I20260924 00:21:01.424376 10629 tablet_service.cc:1467] Processing CreateTablet for tablet 2499754460e04c27ac53d52a4a67f557 (DEFAULT_TABLE table=default.Table_6cf69012192f4f1087e29db70ae8c282 [id=93c1734de58245f8b1d5ab88ec9f0c8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:01.430737 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ec36d59b5114feea144bcb72794136c" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:01.525609 10629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2499754460e04c27ac53d52a4a67f557. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:21:01.443959 10592 leader_election.cc:341] T 85a0ad61b7cb4e11b30e84ddc4655b64 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:01.468237 10655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ec36d59b5114feea144bcb72794136c" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:01.469949 10656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0074b1184ec4883ab1016330d98c968" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
W20260924 00:21:01.530123 10592 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:35359, user_credentials={real_user=slave}} blocked reactor thread for 86195.4us
I20260924 00:21:01.530769 10656 raft_consensus.cc:2379] T e0074b1184ec4883ab1016330d98c968 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48fbd603abe146c585d15196e1b5be08 in current term 1: Already voted for candidate c8626e300d0644b0908797ad61bd7cef in this term.
I20260924 00:21:01.525427 10654 raft_consensus.cc:2454] T 1ec36d59b5114feea144bcb72794136c P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
I20260924 00:21:01.531893 10592 leader_election.cc:302] T 96960f3061fb43e9939b470f49edf1e9 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670; no voters: c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:01.533026 11049 rpcz_store.cc:276] 0924 00:21:00.112065 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:00.114832 (+ 2767us) service_pool.cc:227] Handling call
0924 00:21:01.012580 (+897748us) ts_tablet_manager.cc:1145] Deleting tablet c83942a74816465fa7f3f2c3e18c98bb
0924 00:21:01.012619 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:01.199897 (+187278us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:01.264611 (+ 64714us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:01.416372 (+151761us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":132,"tablet-delete.queue_time_us":897222}
I20260924 00:21:01.534008 10636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60ebab3e93f54383b87aeb54bbde722d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:01.536919 11218 leader_election.cc:288] T bcd8a6b92b8e492d96994195fa7f3ae5 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
W20260924 00:21:01.532855 10592 leader_election.cc:341] T 85a0ad61b7cb4e11b30e84ddc4655b64 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:01.550392 11049 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 100) took 1408 ms. Trace:
I20260924 00:21:01.553188 11218 raft_consensus.cc:2741] T 1ec36d59b5114feea144bcb72794136c P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260924 00:21:01.553867 10733 leader_election.cc:302] T e0074b1184ec4883ab1016330d98c968 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08; no voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:01.498343 11039 raft_consensus.cc:729] T 96960f3061fb43e9939b470f49edf1e9 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:01.559368 11039 consensus_queue.cc:260] T 96960f3061fb43e9939b470f49edf1e9 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:01.561527 11039 ts_tablet_manager.cc:1428] T 96960f3061fb43e9939b470f49edf1e9 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.152s user 0.005s sys 0.000s
I20260924 00:21:01.562345 11039 tablet_bootstrap.cc:492] T d44932f8f08e4ea7acc05b8ec511f045 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
W20260924 00:21:01.563364 10876 leader_election.cc:341] T bcd8a6b92b8e492d96994195fa7f3ae5 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: bcd8a6b92b8e492d96994195fa7f3ae5
I20260924 00:21:01.568055 11156 log.cc:1196] T af2f58e6757a4b738a8ec38b2e228f9f P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/af2f58e6757a4b738a8ec38b2e228f9f
I20260924 00:21:01.568813 11156 ts_tablet_manager.cc:1939] T af2f58e6757a4b738a8ec38b2e228f9f P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:01.571063 11049 rpcz_store.cc:276] 0924 00:21:00.142041 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:00.147861 (+ 5820us) service_pool.cc:227] Handling call
0924 00:21:01.550035 (+1402174us) ts_tablet_manager.cc:1145] Deleting tablet c7e50ac8b7f947b1bfba8cd1cff0ea7e
0924 00:21:01.550076 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:01.550337 (+ 261us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1401658}
I20260924 00:21:01.572152 11049 tablet_replica.cc:330] T eeb11092b803467e9db732468350e852 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
W20260924 00:21:01.572670 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet c7e50ac8b7f947b1bfba8cd1cff0ea7e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c7e50ac8b7f947b1bfba8cd1cff0ea7e already in progress: creating tablet
I20260924 00:21:01.586346 11040 tablet_bootstrap.cc:654] T d44932f8f08e4ea7acc05b8ec511f045 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:01.587363 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet af2f58e6757a4b738a8ec38b2e228f9f (table default.Table_391ed8efc7a54515837c1b79be85d64c [id=f72ddb5979044cda93de75fcb1983ca4]) successfully deleted
I20260924 00:21:01.587635 11156 tablet_replica.cc:330] T eeb11092b803467e9db732468350e852 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:01.588155 11156 raft_consensus.cc:2229] T eeb11092b803467e9db732468350e852 P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:21:01.588644 11156 pending_rounds.cc:62] T eeb11092b803467e9db732468350e852 P 48fbd603abe146c585d15196e1b5be08: Trying to abort 1 pending ops.
W20260924 00:21:01.589015 10361 catalog_manager.cc:4808] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet ae0c9b785195491987dd8fe397d704e0 because the tablet was not found. No further retry: Not found: Tablet not found: ae0c9b785195491987dd8fe397d704e0
I20260924 00:21:01.589607 11255 raft_consensus.cc:3037] T e0074b1184ec4883ab1016330d98c968 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:01.559782 11252 raft_consensus.cc:2781] T 96960f3061fb43e9939b470f49edf1e9 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:01.595341 11252 raft_consensus.cc:482] T 96960f3061fb43e9939b470f49edf1e9 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 00:21:01.577675 10592 consensus_peers.cc:487] T ae0c9b785195491987dd8fe397d704e0 P b10a9082a37b46f09d581944914c1670 -> Peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Couldn't send request to peer 48fbd603abe146c585d15196e1b5be08. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ae0c9b785195491987dd8fe397d704e0. This is attempt 1: this message will repeat every 5th retry.
I20260924 00:21:01.595655 11049 raft_consensus.cc:2229] T eeb11092b803467e9db732468350e852 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:01.596040 11252 raft_consensus.cc:3037] T 96960f3061fb43e9939b470f49edf1e9 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:01.579276 11251 raft_consensus.cc:686] T 85a0ad61b7cb4e11b30e84ddc4655b64 P b10a9082a37b46f09d581944914c1670 [term 1 LEADER]: Becoming Leader. State: Replica: b10a9082a37b46f09d581944914c1670, State: Running, Role: LEADER
I20260924 00:21:01.599493 11251 consensus_queue.cc:237] T 85a0ad61b7cb4e11b30e84ddc4655b64 P b10a9082a37b46f09d581944914c1670 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:01.605299 11150 leader_election.cc:288] T 1ec36d59b5114feea144bcb72794136c P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Requested vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
W20260924 00:21:01.607570 10878 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:35359, user_credentials={real_user=slave}} blocked reactor thread for 154191us
I20260924 00:21:01.609660 10796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcd8a6b92b8e492d96994195fa7f3ae5" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:21:01.610071 10796 raft_consensus.cc:3037] T bcd8a6b92b8e492d96994195fa7f3ae5 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:01.610790 11199 raft_consensus.cc:686] T 1ec36d59b5114feea144bcb72794136c P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: Becoming Leader. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Running, Role: LEADER
I20260924 00:21:01.612176 10372 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 4420cead1f284af8958f189557833098
I20260924 00:21:01.611618 11199 consensus_queue.cc:237] T 1ec36d59b5114feea144bcb72794136c P 48fbd603abe146c585d15196e1b5be08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
W20260924 00:21:01.612323 10370 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.4.65:55561 (request call id 42) took 1197 ms (client timeout 1000 ms). Trace:
I20260924 00:21:01.614223 10372 master_service.cc:946] ReplaceTablet: received request to replace tablet 5d7d52be4f1f4fc29b8172b9fa878f76 from {username='slave'} at 127.0.0.1:47022
W20260924 00:21:01.612653 10370 rpcz_store.cc:269] 0924 00:21:00.414716 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:00.414848 (+ 132us) service_pool.cc:227] Handling call
0924 00:21:01.612252 (+1197404us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 00:21:01.567264 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 00:21:01.567579 (+ 315us) write_op.cc:432] Acquiring schema lock in shared mode
0924 00:21:01.567597 (+ 18us) write_op.cc:435] Acquired schema lock
0924 00:21:01.567760 (+ 163us) tablet.cc:660] Decoding operations
0924 00:21:01.568011 (+ 251us) write_op.cc:620] Acquiring the partition lock for write op
0924 00:21:01.568043 (+ 32us) write_op.cc:641] Partition lock acquired for write op
0924 00:21:01.568063 (+ 20us) tablet.cc:683] Acquiring locks for 1 operations
0924 00:21:01.568185 (+ 122us) tablet.cc:699] Row locks acquired
0924 00:21:01.568197 (+ 12us) write_op.cc:260] PREPARE: finished
0924 00:21:01.568288 (+ 91us) write_op.cc:270] Start()
0924 00:21:01.568359 (+ 71us) write_op.cc:276] Timestamp: P: 1790209261568223 usec, L: 0
0924 00:21:01.568372 (+ 13us) op_driver.cc:331] REPLICATION: starting
0924 00:21:01.568742 (+ 370us) log.cc:842] Serialized 527 byte log entry
0924 00:21:01.606975 (+ 38233us) op_driver.cc:444] REPLICATION: finished
0924 00:21:01.609045 (+ 2070us) write_op.cc:301] APPLY: starting
0924 00:21:01.609089 (+ 44us) tablet.cc:1365] starting BulkCheckPresence
0924 00:21:01.609173 (+ 84us) tablet.cc:1368] finished BulkCheckPresence
0924 00:21:01.609255 (+ 82us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 00:21:01.609381 (+ 126us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 00:21:01.609499 (+ 118us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0924 00:21:01.610114 (+ 615us) write_op.cc:312] APPLY: finished
0924 00:21:01.610294 (+ 180us) log.cc:842] Serialized 23 byte log entry
0924 00:21:01.610407 (+ 113us) write_op.cc:489] Releasing partition, row and schema locks
0924 00:21:01.610465 (+ 58us) write_op.cc:454] Released schema lock
0924 00:21:01.610494 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0924 00:21:01.610641 (+ 147us) write_op.cc:489] Releasing partition, row and schema locks
0924 00:21:01.610665 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":1946,"apply.run_cpu_time_us":1740,"apply.run_wall_time_us":1740,"num_ops":1,"prepare.queue_time_us":202720,"replication_time_us":38538}]]}
I20260924 00:21:01.617818 10796 raft_consensus.cc:2454] T bcd8a6b92b8e492d96994195fa7f3ae5 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 1.
I20260924 00:21:01.618829 10773 tablet_service.cc:1514] Processing DeleteTablet for tablet 4420cead1f284af8958f189557833098 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:01 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:01.621328 10878 leader_election.cc:302] T bcd8a6b92b8e492d96994195fa7f3ae5 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters: b10a9082a37b46f09d581944914c1670
I20260924 00:21:01.622123 10374 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 1ec36d59b5114feea144bcb72794136c (table default.Table_b1726c09065e4fbaa2e6799b3ebdbc74 [id=4dbb68e0a32349ad8da2945db650584d]). Expected version 1 got 0
I20260924 00:21:01.588832 11156 pending_rounds.cc:68] T eeb11092b803467e9db732468350e852 P 48fbd603abe146c585d15196e1b5be08: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332697133607866368 op_type: NO_OP noop_request { }
I20260924 00:21:01.625973 11156 raft_consensus.cc:2866] T eeb11092b803467e9db732468350e852 P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260924 00:21:01.626336 11156 raft_consensus.cc:2258] T eeb11092b803467e9db732468350e852 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:01.434414 11198 leader_election.cc:288] T 85a0ad61b7cb4e11b30e84ddc4655b64 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Requested vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:01.586498 11041 tablet_bootstrap.cc:492] T 666755ec87bb496db75e43ec9204ec5b P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:01.626907 10366 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet 1ec36d59b5114feea144bcb72794136c (table default.Table_b1726c09065e4fbaa2e6799b3ebdbc74 [id=4dbb68e0a32349ad8da2945db650584d]). Expected version 1 got 0
I20260924 00:21:01.627535 11041 ts_tablet_manager.cc:1397] T 666755ec87bb496db75e43ec9204ec5b P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.301s user 0.008s sys 0.001s
I20260924 00:21:01.627697 10366 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet 96960f3061fb43e9939b470f49edf1e9 (table default.Table_da80d8b26b9b48c79a00dd08f123f4f4 [id=6251651eb5944ee086dd564d86204148]). Expected version 1 got 0
I20260924 00:21:01.629799 11255 raft_consensus.cc:2726] T e0074b1184ec4883ab1016330d98c968 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:01.630029 11041 raft_consensus.cc:348] T 666755ec87bb496db75e43ec9204ec5b P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:01.630656 11041 raft_consensus.cc:374] T 666755ec87bb496db75e43ec9204ec5b P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:01.630891 11041 raft_consensus.cc:729] T 666755ec87bb496db75e43ec9204ec5b P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:01.631492 11041 consensus_queue.cc:260] T 666755ec87bb496db75e43ec9204ec5b P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:01.596060 11049 raft_consensus.cc:2258] T eeb11092b803467e9db732468350e852 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:01.606447 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ec36d59b5114feea144bcb72794136c" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:21:01.632458 10654 raft_consensus.cc:3037] T 1ec36d59b5114feea144bcb72794136c P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:01.638799 10654 raft_consensus.cc:2454] T 1ec36d59b5114feea144bcb72794136c P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 1.
I20260924 00:21:01.630591 11040 tablet_bootstrap.cc:492] T d44932f8f08e4ea7acc05b8ec511f045 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:01.640551 11040 ts_tablet_manager.cc:1397] T d44932f8f08e4ea7acc05b8ec511f045 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.200s user 0.013s sys 0.000s
I20260924 00:21:01.640389 10372 catalog_manager.cc:5495] T 1ec36d59b5114feea144bcb72794136c P 48fbd603abe146c585d15196e1b5be08 reported cstate change: term changed from 0 to 1, leader changed from <none> to 48fbd603abe146c585d15196e1b5be08 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "48fbd603abe146c585d15196e1b5be08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } health_report { overall_health: UNKNOWN } } }
I20260924 00:21:01.641953 10372 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet 1ec36d59b5114feea144bcb72794136c (table default.Table_b1726c09065e4fbaa2e6799b3ebdbc74 [id=4dbb68e0a32349ad8da2945db650584d]). Expected version 1 got 0
I20260924 00:21:01.632061 11156 ts_tablet_manager.cc:1905] T eeb11092b803467e9db732468350e852 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:01.642720 10372 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet 96960f3061fb43e9939b470f49edf1e9 (table default.Table_da80d8b26b9b48c79a00dd08f123f4f4 [id=6251651eb5944ee086dd564d86204148]). Expected version 1 got 0
I20260924 00:21:01.622787 10915 tablet_service.cc:1514] Processing DeleteTablet for tablet 4420cead1f284af8958f189557833098 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:01 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:01.644526 11256 raft_consensus.cc:2781] T bcd8a6b92b8e492d96994195fa7f3ae5 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:01.645035 11256 raft_consensus.cc:686] T bcd8a6b92b8e492d96994195fa7f3ae5 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Becoming Leader. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Running, Role: LEADER
I20260924 00:21:01.646176 11256 consensus_queue.cc:237] T bcd8a6b92b8e492d96994195fa7f3ae5 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
W20260924 00:21:01.647629 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 4420cead1f284af8958f189557833098 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4420cead1f284af8958f189557833098 already in progress: creating tablet
I20260924 00:21:01.647143 10915 tablet_service.cc:1514] Processing DeleteTablet for tablet eeb11092b803467e9db732468350e852 with delete_type TABLET_DATA_DELETED (replaced by tablet 4325b60efc1649c9ba4828d0a098a676) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:01.650178 11051 tablet_replica.cc:330] T eeb11092b803467e9db732468350e852 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:01.650906 11051 raft_consensus.cc:2229] T eeb11092b803467e9db732468350e852 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:01.651309 11051 raft_consensus.cc:2258] T eeb11092b803467e9db732468350e852 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:01.651427 11252 raft_consensus.cc:504] T 96960f3061fb43e9939b470f49edf1e9 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:01.652809 11051 ts_tablet_manager.cc:1905] T eeb11092b803467e9db732468350e852 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:01.653724 10774 tablet_service.cc:1514] Processing DeleteTablet for tablet bcd8a6b92b8e492d96994195fa7f3ae5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:58 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:01.654556 10796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96960f3061fb43e9939b470f49edf1e9" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:21:01.617805 10635 tablet_service.cc:1514] Processing DeleteTablet for tablet 4420cead1f284af8958f189557833098 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:01 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:01.623579 11260 consensus_queue.cc:1035] T ccb26e0fbc1948bc9a00e9e26c4d1139 P b10a9082a37b46f09d581944914c1670 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 00:21:01.655117 10796 raft_consensus.cc:3037] T 96960f3061fb43e9939b470f49edf1e9 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:01.647564 10922 tablet_service.cc:1514] Processing DeleteTablet for tablet bcd8a6b92b8e492d96994195fa7f3ae5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:58 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:01.662426 10796 raft_consensus.cc:2454] T 96960f3061fb43e9939b470f49edf1e9 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b10a9082a37b46f09d581944914c1670 in term 1.
I20260924 00:21:01.663516 10592 leader_election.cc:302] T 96960f3061fb43e9939b470f49edf1e9 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670; no voters:
I20260924 00:21:01.667547 11049 ts_tablet_manager.cc:1905] T eeb11092b803467e9db732468350e852 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:01.671629 11260 raft_consensus.cc:2781] T 96960f3061fb43e9939b470f49edf1e9 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:01.674481 11040 raft_consensus.cc:348] T d44932f8f08e4ea7acc05b8ec511f045 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:01.675199 11040 raft_consensus.cc:374] T d44932f8f08e4ea7acc05b8ec511f045 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:01.675454 11040 raft_consensus.cc:729] T d44932f8f08e4ea7acc05b8ec511f045 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:01.675938 11260 raft_consensus.cc:686] T 96960f3061fb43e9939b470f49edf1e9 P b10a9082a37b46f09d581944914c1670 [term 1 LEADER]: Becoming Leader. State: Replica: b10a9082a37b46f09d581944914c1670, State: Running, Role: LEADER
I20260924 00:21:01.676769 11260 consensus_queue.cc:237] T 96960f3061fb43e9939b470f49edf1e9 P b10a9082a37b46f09d581944914c1670 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:01.677209 11040 consensus_queue.cc:260] T d44932f8f08e4ea7acc05b8ec511f045 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:01.682030 11040 ts_tablet_manager.cc:1428] T d44932f8f08e4ea7acc05b8ec511f045 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.041s user 0.005s sys 0.000s
I20260924 00:21:01.693907 11039 tablet_bootstrap.cc:654] T d44932f8f08e4ea7acc05b8ec511f045 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:01.707026 11040 tablet_bootstrap.cc:492] T 85a0ad61b7cb4e11b30e84ddc4655b64 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:01.679296 11041 ts_tablet_manager.cc:1428] T 666755ec87bb496db75e43ec9204ec5b P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.052s user 0.004s sys 0.000s
I20260924 00:21:01.715538 11051 ts_tablet_manager.cc:1918] T eeb11092b803467e9db732468350e852 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:01.715914 11051 log.cc:1196] T eeb11092b803467e9db732468350e852 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/eeb11092b803467e9db732468350e852
I20260924 00:21:01.716511 11051 ts_tablet_manager.cc:1939] T eeb11092b803467e9db732468350e852 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:01.717020 11041 tablet_bootstrap.cc:492] T 6ac9c7da7fb546f6ae935371799cbad6 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
W20260924 00:21:01.726593 10365 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 119d83562ef546cab496cd7b44fecc15 consensus state has no leader
I20260924 00:21:01.732501 10366 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_0e5a147af0824c1d888bee1c066cb368" } new_table_name: "default.Table_4a5edca73a094b4d93ba4d389679ab48" modify_external_catalogs: true
I20260924 00:21:01.733911 10373 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 666755ec87bb496db75e43ec9204ec5b
I20260924 00:21:01.734392 10365 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_e94c810f23d34f62bfa2bb1a7e5f0a02"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:01.735957 10365 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e94c810f23d34f62bfa2bb1a7e5f0a02 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:01.736927 10635 tablet_service.cc:1514] Processing DeleteTablet for tablet 666755ec87bb496db75e43ec9204ec5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:01 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:01.740602 10373 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_4c54324167a8434a90e8dcb1354111f9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:01.741938 11051 tablet_replica.cc:330] T bcd8a6b92b8e492d96994195fa7f3ae5 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
W20260924 00:21:01.742069 10373 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4c54324167a8434a90e8dcb1354111f9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:01.742587 11051 raft_consensus.cc:2229] T bcd8a6b92b8e492d96994195fa7f3ae5 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:21:01.743187 11051 pending_rounds.cc:62] T bcd8a6b92b8e492d96994195fa7f3ae5 P c8626e300d0644b0908797ad61bd7cef: Trying to abort 1 pending ops.
I20260924 00:21:01.743395 11051 pending_rounds.cc:68] T bcd8a6b92b8e492d96994195fa7f3ae5 P c8626e300d0644b0908797ad61bd7cef: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332697135824900096 op_type: NO_OP noop_request { }
I20260924 00:21:01.743727 11051 raft_consensus.cc:2866] T bcd8a6b92b8e492d96994195fa7f3ae5 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260924 00:21:01.744014 11051 raft_consensus.cc:2258] T bcd8a6b92b8e492d96994195fa7f3ae5 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:01.744689 10921 tablet_service.cc:1514] Processing DeleteTablet for tablet 666755ec87bb496db75e43ec9204ec5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:01 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:01.747015 11156 ts_tablet_manager.cc:1918] T eeb11092b803467e9db732468350e852 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:01.744606 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet eeb11092b803467e9db732468350e852 (table default.Table_0e5a147af0824c1d888bee1c066cb368 [id=7a76983970cd491a8e8635db379bdd10]) successfully deleted
I20260924 00:21:01.747531 11156 log.cc:1196] T eeb11092b803467e9db732468350e852 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/eeb11092b803467e9db732468350e852
I20260924 00:21:01.751185 10940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96960f3061fb43e9939b470f49edf1e9" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef"
I20260924 00:21:01.751821 10940 raft_consensus.cc:3037] T 96960f3061fb43e9939b470f49edf1e9 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:01.754654 10778 tablet_service.cc:1514] Processing DeleteTablet for tablet 666755ec87bb496db75e43ec9204ec5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:01 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:01.754729 11252 leader_election.cc:288] T 96960f3061fb43e9939b470f49edf1e9 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Requested vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:01.759358 10940 raft_consensus.cc:2454] T 96960f3061fb43e9939b470f49edf1e9 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b10a9082a37b46f09d581944914c1670 in term 1.
I20260924 00:21:01.761377 11051 ts_tablet_manager.cc:1905] T bcd8a6b92b8e492d96994195fa7f3ae5 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:01.761099 11256 consensus_queue.cc:1035] T e0074b1184ec4883ab1016330d98c968 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Connected to new peer: Peer: permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 00:21:01.765627 11156 ts_tablet_manager.cc:1939] T eeb11092b803467e9db732468350e852 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:01.768253 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet eeb11092b803467e9db732468350e852 (table default.Table_0e5a147af0824c1d888bee1c066cb368 [id=7a76983970cd491a8e8635db379bdd10]) successfully deleted
W20260924 00:21:01.774734 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 4420cead1f284af8958f189557833098 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4420cead1f284af8958f189557833098 already in progress: creating tablet
I20260924 00:21:01.776484 11156 tablet_replica.cc:330] T bcd8a6b92b8e492d96994195fa7f3ae5 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
W20260924 00:21:01.778110 10589 consensus_peers.cc:487] T ae0c9b785195491987dd8fe397d704e0 P b10a9082a37b46f09d581944914c1670 -> Peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Couldn't send request to peer c8626e300d0644b0908797ad61bd7cef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ae0c9b785195491987dd8fe397d704e0. This is attempt 1: this message will repeat every 5th retry.
I20260924 00:21:01.780526 11156 raft_consensus.cc:2229] T bcd8a6b92b8e492d96994195fa7f3ae5 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:01.785629 11156 raft_consensus.cc:2258] T bcd8a6b92b8e492d96994195fa7f3ae5 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:01.795814 11156 ts_tablet_manager.cc:1905] T bcd8a6b92b8e492d96994195fa7f3ae5 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:01.808050 10367 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet ccb26e0fbc1948bc9a00e9e26c4d1139
I20260924 00:21:01.809476 10367 catalog_manager.cc:6565] ReplaceTablet: tablet ccb26e0fbc1948bc9a00e9e26c4d1139 deleted and replaced by tablet f9c6a39b2e2842c59b0f996f39d73162
I20260924 00:21:01.809177 11256 consensus_queue.cc:1035] T e0074b1184ec4883ab1016330d98c968 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Connected to new peer: Peer: permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:01.810497 10778 tablet_service.cc:1514] Processing DeleteTablet for tablet ccb26e0fbc1948bc9a00e9e26c4d1139 with delete_type TABLET_DATA_DELETED (replaced by tablet f9c6a39b2e2842c59b0f996f39d73162) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:01.811764 10921 tablet_service.cc:1514] Processing DeleteTablet for tablet ccb26e0fbc1948bc9a00e9e26c4d1139 with delete_type TABLET_DATA_DELETED (replaced by tablet f9c6a39b2e2842c59b0f996f39d73162) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:01.817502 10635 tablet_service.cc:1514] Processing DeleteTablet for tablet ccb26e0fbc1948bc9a00e9e26c4d1139 with delete_type TABLET_DATA_DELETED (replaced by tablet f9c6a39b2e2842c59b0f996f39d73162) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:01.818231 10367 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_01b5209b9d5c4115b0bf6aef48b51bd8" } modify_external_catalogs: true
I20260924 00:21:01.819453 10367 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_01b5209b9d5c4115b0bf6aef48b51bd8" } modify_external_catalogs: true
I20260924 00:21:01.841806 11041 tablet_bootstrap.cc:700] T 6ac9c7da7fb546f6ae935371799cbad6 P b10a9082a37b46f09d581944914c1670: Time spent opening tablet: real 0.117s user 0.004s sys 0.000s
I20260924 00:21:01.842492 11041 tablet_bootstrap.cc:654] T 6ac9c7da7fb546f6ae935371799cbad6 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:01.844470 11039 tablet_bootstrap.cc:492] T d44932f8f08e4ea7acc05b8ec511f045 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:01.844853 11039 ts_tablet_manager.cc:1397] T d44932f8f08e4ea7acc05b8ec511f045 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.283s user 0.006s sys 0.010s
I20260924 00:21:01.848873 11039 raft_consensus.cc:348] T d44932f8f08e4ea7acc05b8ec511f045 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:01.849576 11039 raft_consensus.cc:374] T d44932f8f08e4ea7acc05b8ec511f045 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:01.849805 11039 raft_consensus.cc:729] T d44932f8f08e4ea7acc05b8ec511f045 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:01.850486 11039 consensus_queue.cc:260] T d44932f8f08e4ea7acc05b8ec511f045 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:01.852725 11039 ts_tablet_manager.cc:1428] T d44932f8f08e4ea7acc05b8ec511f045 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20260924 00:21:01.858752 10374 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet f294831873d548e4b9201c1bbb964096 (table default.Table_de36c9f702024914965bc6b371eda68e [id=e2cdad6dfe7846ebb3c14f5611cf0f18]). Expected version 1 got 0
I20260924 00:21:01.859515 10374 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 85a0ad61b7cb4e11b30e84ddc4655b64 (table default.Table_67493ad72aac4bbca26a6a322a60e5cd [id=12285914811f4220822b47a2a62b01ab]). Expected version 2 got 0
I20260924 00:21:01.860092 10374 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 96960f3061fb43e9939b470f49edf1e9 (table default.Table_da80d8b26b9b48c79a00dd08f123f4f4 [id=6251651eb5944ee086dd564d86204148]). Expected version 1 got 0
I20260924 00:21:01.864663 11051 ts_tablet_manager.cc:1918] T bcd8a6b92b8e492d96994195fa7f3ae5 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:01.865072 11051 log.cc:1196] T bcd8a6b92b8e492d96994195fa7f3ae5 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/bcd8a6b92b8e492d96994195fa7f3ae5
I20260924 00:21:01.865774 11051 ts_tablet_manager.cc:1939] T bcd8a6b92b8e492d96994195fa7f3ae5 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:01.868191 11039 tablet_bootstrap.cc:492] T 5d7d52be4f1f4fc29b8172b9fa878f76 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:01.879494 10372 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_3078c59b2af247ab91ea35e4f8bf9b9c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:01.883584 10369 master_service.cc:946] ReplaceTablet: received request to replace tablet 95a7a1d2147c4d7ea63ca37a8f0ea4e6 from {username='slave'} at 127.0.0.1:47022
W20260924 00:21:01.886605 10372 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3078c59b2af247ab91ea35e4f8bf9b9c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:01.901098 11049 ts_tablet_manager.cc:1918] T eeb11092b803467e9db732468350e852 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:01.901455 11049 log.cc:1196] T eeb11092b803467e9db732468350e852 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/eeb11092b803467e9db732468350e852
I20260924 00:21:01.902027 11049 ts_tablet_manager.cc:1939] T eeb11092b803467e9db732468350e852 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:01.902253 11051 tablet_replica.cc:330] T ccb26e0fbc1948bc9a00e9e26c4d1139 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:01.903723 11049 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 106) took 1662 ms. Trace:
I20260924 00:21:01.904634 11039 tablet_bootstrap.cc:654] T 5d7d52be4f1f4fc29b8172b9fa878f76 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:01.905606 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet bcd8a6b92b8e492d96994195fa7f3ae5 (table default.Table_99da18207fc542fbb68972724c4d410e [id=f75c1da3bd734e92bc105c473236fa9b]) successfully deleted
W20260924 00:21:01.906391 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 666755ec87bb496db75e43ec9204ec5b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 666755ec87bb496db75e43ec9204ec5b already in progress: creating tablet
I20260924 00:21:01.908797 11051 raft_consensus.cc:2229] T ccb26e0fbc1948bc9a00e9e26c4d1139 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:01.909274 11051 raft_consensus.cc:2258] T ccb26e0fbc1948bc9a00e9e26c4d1139 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:01.911141 11041 tablet_bootstrap.cc:492] T 6ac9c7da7fb546f6ae935371799cbad6 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:01.911499 11041 ts_tablet_manager.cc:1397] T 6ac9c7da7fb546f6ae935371799cbad6 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.195s user 0.009s sys 0.000s
I20260924 00:21:01.913650 11041 raft_consensus.cc:348] T 6ac9c7da7fb546f6ae935371799cbad6 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:01.914323 11041 raft_consensus.cc:374] T 6ac9c7da7fb546f6ae935371799cbad6 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:01.915334 11260 raft_consensus.cc:482] T 5b0345252c3c496db2987c30adce3da7 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:01.915738 11260 raft_consensus.cc:504] T 5b0345252c3c496db2987c30adce3da7 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:01.919759 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b0345252c3c496db2987c30adce3da7" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
W20260924 00:21:01.921141 10366 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4325b60efc1649c9ba4828d0a098a676 consensus state has no leader
W20260924 00:21:01.921634 10589 leader_election.cc:341] T 5b0345252c3c496db2987c30adce3da7 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:01.922180 11260 leader_election.cc:288] T 5b0345252c3c496db2987c30adce3da7 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:01.924046 11051 ts_tablet_manager.cc:1905] T ccb26e0fbc1948bc9a00e9e26c4d1139 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:01.914554 11041 raft_consensus.cc:729] T 6ac9c7da7fb546f6ae935371799cbad6 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:01.927013 10796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b0345252c3c496db2987c30adce3da7" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:01.928761 10592 leader_election.cc:341] T 5b0345252c3c496db2987c30adce3da7 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:01.928670 11041 consensus_queue.cc:260] T 6ac9c7da7fb546f6ae935371799cbad6 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:01.929157 10592 leader_election.cc:302] T 5b0345252c3c496db2987c30adce3da7 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:01.930104 10366 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_610074e75ae343618ae2141e342529b9" } new_table_name: "default.Table_d01ccf505bd94805b62c8f0755dad1d1" modify_external_catalogs: true
I20260924 00:21:01.903985 11049 rpcz_store.cc:276] 0924 00:21:00.241669 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:00.241878 (+ 209us) service_pool.cc:227] Handling call
0924 00:21:01.572015 (+1330137us) ts_tablet_manager.cc:1145] Deleting tablet eeb11092b803467e9db732468350e852
0924 00:21:01.572055 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:01.729199 (+157144us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:01.900837 (+171638us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:01.903646 (+ 2809us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":24820,"tablet-delete.queue_time_us":1329442}
I20260924 00:21:01.932263 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet eeb11092b803467e9db732468350e852 (table default.Table_4a5edca73a094b4d93ba4d389679ab48 [id=7a76983970cd491a8e8635db379bdd10]) successfully deleted
I20260924 00:21:01.940419 11049 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 107) took 1684 ms. Trace:
I20260924 00:21:01.940737 11049 rpcz_store.cc:276] 0924 00:21:00.256160 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:00.256307 (+ 147us) service_pool.cc:227] Handling call
0924 00:21:01.940069 (+1683762us) ts_tablet_manager.cc:1145] Deleting tablet ffa011cf8dc446de849bea75a4f20056
0924 00:21:01.940111 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:01.940361 (+ 250us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1683225}
W20260924 00:21:01.941887 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet ffa011cf8dc446de849bea75a4f20056 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ffa011cf8dc446de849bea75a4f20056 already in progress: creating tablet
I20260924 00:21:01.942284 10373 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_fd43d6016c81497586729ff4cbe3213c" } modify_external_catalogs: true
I20260924 00:21:01.942839 10373 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_fd43d6016c81497586729ff4cbe3213c" } modify_external_catalogs: true
I20260924 00:21:01.943156 11252 raft_consensus.cc:2726] T 5b0345252c3c496db2987c30adce3da7 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:01.955153 11039 tablet_bootstrap.cc:492] T 5d7d52be4f1f4fc29b8172b9fa878f76 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:01.955941 11039 ts_tablet_manager.cc:1397] T 5d7d52be4f1f4fc29b8172b9fa878f76 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.088s user 0.010s sys 0.000s
I20260924 00:21:01.956187 10369 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_294eebbf563b4b24a4f75064b4e9233b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:01.958504 11039 raft_consensus.cc:348] T 5d7d52be4f1f4fc29b8172b9fa878f76 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:01.959220 11039 raft_consensus.cc:374] T 5d7d52be4f1f4fc29b8172b9fa878f76 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:01.959470 11039 raft_consensus.cc:729] T 5d7d52be4f1f4fc29b8172b9fa878f76 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:01.960012 11039 consensus_queue.cc:260] T 5d7d52be4f1f4fc29b8172b9fa878f76 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:01.963517 11260 raft_consensus.cc:482] T fcf651adb1d3459facfe3a8df2a91ed7 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:01.963951 11260 raft_consensus.cc:504] T fcf651adb1d3459facfe3a8df2a91ed7 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:01.969161 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcf651adb1d3459facfe3a8df2a91ed7" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
W20260924 00:21:01.970674 10589 leader_election.cc:341] T fcf651adb1d3459facfe3a8df2a91ed7 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:01.971835 11260 leader_election.cc:288] T fcf651adb1d3459facfe3a8df2a91ed7 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:21:01.973404 10796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcf651adb1d3459facfe3a8df2a91ed7" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:01.974733 10592 leader_election.cc:341] T fcf651adb1d3459facfe3a8df2a91ed7 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:01.975095 10592 leader_election.cc:302] T fcf651adb1d3459facfe3a8df2a91ed7 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:01.981989 11260 raft_consensus.cc:2726] T fcf651adb1d3459facfe3a8df2a91ed7 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260924 00:21:01.982564 10369 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_294eebbf563b4b24a4f75064b4e9233b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:01.986354 11040 tablet_bootstrap.cc:654] T 85a0ad61b7cb4e11b30e84ddc4655b64 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:02.000813 11039 ts_tablet_manager.cc:1428] T 5d7d52be4f1f4fc29b8172b9fa878f76 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.044s user 0.005s sys 0.000s
I20260924 00:21:02.001854 11039 tablet_bootstrap.cc:492] T 86374d72238d42c29542e649a7a0200d P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:02.006145 11156 ts_tablet_manager.cc:1918] T bcd8a6b92b8e492d96994195fa7f3ae5 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:02.006668 11156 log.cc:1196] T bcd8a6b92b8e492d96994195fa7f3ae5 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/bcd8a6b92b8e492d96994195fa7f3ae5
I20260924 00:21:02.007730 11156 ts_tablet_manager.cc:1939] T bcd8a6b92b8e492d96994195fa7f3ae5 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:02.008091 11041 ts_tablet_manager.cc:1428] T 6ac9c7da7fb546f6ae935371799cbad6 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.096s user 0.006s sys 0.000s
I20260924 00:21:02.011153 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet bcd8a6b92b8e492d96994195fa7f3ae5 (table default.Table_99da18207fc542fbb68972724c4d410e [id=f75c1da3bd734e92bc105c473236fa9b]) successfully deleted
I20260924 00:21:02.022941 11049 tablet_replica.cc:330] T 6ac9c7da7fb546f6ae935371799cbad6 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:02.023052 11051 ts_tablet_manager.cc:1918] T ccb26e0fbc1948bc9a00e9e26c4d1139 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:02.023407 11051 log.cc:1196] T ccb26e0fbc1948bc9a00e9e26c4d1139 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/ccb26e0fbc1948bc9a00e9e26c4d1139
I20260924 00:21:02.028087 11049 raft_consensus.cc:2229] T 6ac9c7da7fb546f6ae935371799cbad6 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:02.029888 11049 raft_consensus.cc:2258] T 6ac9c7da7fb546f6ae935371799cbad6 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:02.031504 11049 ts_tablet_manager.cc:1905] T 6ac9c7da7fb546f6ae935371799cbad6 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:02.033100 10365 master_service.cc:946] ReplaceTablet: received request to replace tablet f3432ac96da94d5099bcf9764c2b32e1 from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:02.035585 11283 tablet.cc:1722] T e0074b1184ec4883ab1016330d98c968 P b10a9082a37b46f09d581944914c1670: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260924 00:21:02.039999 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 666755ec87bb496db75e43ec9204ec5b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 666755ec87bb496db75e43ec9204ec5b already in progress: creating tablet
I20260924 00:21:02.041767 11040 tablet_bootstrap.cc:492] T 85a0ad61b7cb4e11b30e84ddc4655b64 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:02.044652 11039 tablet_bootstrap.cc:654] T 86374d72238d42c29542e649a7a0200d P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:02.045533 11156 tablet_replica.cc:330] T ccb26e0fbc1948bc9a00e9e26c4d1139 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:02.045400 11286 tablet.cc:1722] T e0074b1184ec4883ab1016330d98c968 P 48fbd603abe146c585d15196e1b5be08: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260924 00:21:02.046383 11156 raft_consensus.cc:2229] T ccb26e0fbc1948bc9a00e9e26c4d1139 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:02.046983 11156 raft_consensus.cc:2258] T ccb26e0fbc1948bc9a00e9e26c4d1139 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:02.048087 11282 tablet.cc:1722] T e0074b1184ec4883ab1016330d98c968 P c8626e300d0644b0908797ad61bd7cef: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260924 00:21:02.050083 11040 ts_tablet_manager.cc:1397] T 85a0ad61b7cb4e11b30e84ddc4655b64 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.367s user 0.011s sys 0.008s
I20260924 00:21:02.024061 11051 ts_tablet_manager.cc:1939] T ccb26e0fbc1948bc9a00e9e26c4d1139 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:02.059038 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet ccb26e0fbc1948bc9a00e9e26c4d1139 (table default.Table_01b5209b9d5c4115b0bf6aef48b51bd8 [id=7e9e663acedf4a1fae2f04cde97b7907]) successfully deleted
I20260924 00:21:02.063100 11156 ts_tablet_manager.cc:1905] T ccb26e0fbc1948bc9a00e9e26c4d1139 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:02.052434 11040 raft_consensus.cc:348] T 85a0ad61b7cb4e11b30e84ddc4655b64 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:02.068728 11040 raft_consensus.cc:374] T 85a0ad61b7cb4e11b30e84ddc4655b64 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:02.069041 11040 raft_consensus.cc:729] T 85a0ad61b7cb4e11b30e84ddc4655b64 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:02.075103 11039 tablet_bootstrap.cc:492] T 86374d72238d42c29542e649a7a0200d P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:02.075448 11041 tablet_bootstrap.cc:492] T 195b033091b543b58ee08417eae2c0e6 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:02.075511 11039 ts_tablet_manager.cc:1397] T 86374d72238d42c29542e649a7a0200d P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.074s user 0.007s sys 0.001s
I20260924 00:21:02.076306 11254 raft_consensus.cc:482] T f294831873d548e4b9201c1bbb964096 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:02.076825 11254 raft_consensus.cc:504] T f294831873d548e4b9201c1bbb964096 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:02.077626 11039 raft_consensus.cc:348] T 86374d72238d42c29542e649a7a0200d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:02.079456 11039 raft_consensus.cc:374] T 86374d72238d42c29542e649a7a0200d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:02.079692 11039 raft_consensus.cc:729] T 86374d72238d42c29542e649a7a0200d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:02.079852 10798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f294831873d548e4b9201c1bbb964096" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:02.080328 11041 tablet_bootstrap.cc:654] T 195b033091b543b58ee08417eae2c0e6 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:02.080243 11039 consensus_queue.cc:260] T 86374d72238d42c29542e649a7a0200d P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:02.080674 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f294831873d548e4b9201c1bbb964096" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
W20260924 00:21:02.081290 10878 leader_election.cc:341] T f294831873d548e4b9201c1bbb964096 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:02.081434 10654 raft_consensus.cc:2454] T f294831873d548e4b9201c1bbb964096 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
I20260924 00:21:02.082640 10876 leader_election.cc:302] T f294831873d548e4b9201c1bbb964096 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08
I20260924 00:21:02.083581 11254 leader_election.cc:288] T f294831873d548e4b9201c1bbb964096 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:02.069898 11040 consensus_queue.cc:260] T 85a0ad61b7cb4e11b30e84ddc4655b64 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:02.094864 11040 ts_tablet_manager.cc:1428] T 85a0ad61b7cb4e11b30e84ddc4655b64 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.044s user 0.004s sys 0.000s
I20260924 00:21:02.095919 11256 raft_consensus.cc:2781] T f294831873d548e4b9201c1bbb964096 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:02.096257 11256 raft_consensus.cc:482] T f294831873d548e4b9201c1bbb964096 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:02.096571 11256 raft_consensus.cc:3037] T f294831873d548e4b9201c1bbb964096 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:02.103914 11256 raft_consensus.cc:504] T f294831873d548e4b9201c1bbb964096 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:02.106130 11039 ts_tablet_manager.cc:1428] T 86374d72238d42c29542e649a7a0200d P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.030s user 0.006s sys 0.000s
I20260924 00:21:02.106986 11039 tablet_bootstrap.cc:492] T 5b0345252c3c496db2987c30adce3da7 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:02.107203 11040 tablet_bootstrap.cc:492] T 86374d72238d42c29542e649a7a0200d P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:02.111027 11255 raft_consensus.cc:482] T 85a0ad61b7cb4e11b30e84ddc4655b64 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:02.111497 11255 raft_consensus.cc:504] T 85a0ad61b7cb4e11b30e84ddc4655b64 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:02.113034 11256 leader_election.cc:288] T f294831873d548e4b9201c1bbb964096 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:02.112882 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f294831873d548e4b9201c1bbb964096" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:21:02.113472 11255 leader_election.cc:288] T 85a0ad61b7cb4e11b30e84ddc4655b64 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:02.113513 10654 raft_consensus.cc:3037] T f294831873d548e4b9201c1bbb964096 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:02.114850 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85a0ad61b7cb4e11b30e84ddc4655b64" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:02.115574 10942 raft_consensus.cc:2379] T 85a0ad61b7cb4e11b30e84ddc4655b64 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48fbd603abe146c585d15196e1b5be08 in current term 1: Already voted for candidate b10a9082a37b46f09d581944914c1670 in this term.
I20260924 00:21:02.116843 10798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f294831873d548e4b9201c1bbb964096" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:21:02.118113 10656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85a0ad61b7cb4e11b30e84ddc4655b64" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:02.120378 10733 leader_election.cc:302] T 85a0ad61b7cb4e11b30e84ddc4655b64 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08; no voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef
W20260924 00:21:02.121614 10878 leader_election.cc:341] T f294831873d548e4b9201c1bbb964096 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:02.121671 10369 master_service.cc:946] ReplaceTablet: received request to replace tablet b06bf1e4691441769e162aa5404d4656 from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:02.122233 11255 raft_consensus.cc:3037] T 85a0ad61b7cb4e11b30e84ddc4655b64 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:02.122220 10654 raft_consensus.cc:2454] T f294831873d548e4b9201c1bbb964096 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 1.
I20260924 00:21:02.123301 10876 leader_election.cc:302] T f294831873d548e4b9201c1bbb964096 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08
I20260924 00:21:02.124516 11256 raft_consensus.cc:2781] T f294831873d548e4b9201c1bbb964096 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:02.124998 11256 raft_consensus.cc:686] T f294831873d548e4b9201c1bbb964096 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Becoming Leader. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Running, Role: LEADER
I20260924 00:21:02.125787 11256 consensus_queue.cc:237] T f294831873d548e4b9201c1bbb964096 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:02.127074 10372 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_ab9660d8c1a5491c8fee2f43483105a7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:02.128486 10372 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ab9660d8c1a5491c8fee2f43483105a7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:02.129344 11255 raft_consensus.cc:2726] T 85a0ad61b7cb4e11b30e84ddc4655b64 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:02.133476 10369 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_13a88cf182fe497d84366126f2fdd4ee" } new_table_name: "default.Table_ee73693b7a6e46a3af53f7828da00f9e" modify_external_catalogs: true
I20260924 00:21:02.139729 11039 tablet_bootstrap.cc:654] T 5b0345252c3c496db2987c30adce3da7 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:02.150486 11049 ts_tablet_manager.cc:1918] T 6ac9c7da7fb546f6ae935371799cbad6 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:02.150840 11049 log.cc:1196] T 6ac9c7da7fb546f6ae935371799cbad6 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/6ac9c7da7fb546f6ae935371799cbad6
I20260924 00:21:02.151429 11049 ts_tablet_manager.cc:1939] T 6ac9c7da7fb546f6ae935371799cbad6 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:02.152123 11049 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 108) took 1754 ms. Trace:
I20260924 00:21:02.146937 11156 ts_tablet_manager.cc:1918] T ccb26e0fbc1948bc9a00e9e26c4d1139 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:02.153234 11156 log.cc:1196] T ccb26e0fbc1948bc9a00e9e26c4d1139 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/ccb26e0fbc1948bc9a00e9e26c4d1139
I20260924 00:21:02.153977 11156 ts_tablet_manager.cc:1939] T ccb26e0fbc1948bc9a00e9e26c4d1139 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:02.156085 11255 raft_consensus.cc:482] T d44932f8f08e4ea7acc05b8ec511f045 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:02.156500 11255 raft_consensus.cc:504] T d44932f8f08e4ea7acc05b8ec511f045 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:02.159332 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d44932f8f08e4ea7acc05b8ec511f045" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:02.159924 10942 raft_consensus.cc:2454] T d44932f8f08e4ea7acc05b8ec511f045 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 0.
I20260924 00:21:02.159621 11255 leader_election.cc:288] T d44932f8f08e4ea7acc05b8ec511f045 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:02.161486 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d44932f8f08e4ea7acc05b8ec511f045" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:02.162122 10654 raft_consensus.cc:2454] T d44932f8f08e4ea7acc05b8ec511f045 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 0.
I20260924 00:21:02.163426 10732 leader_election.cc:302] T d44932f8f08e4ea7acc05b8ec511f045 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:02.164745 11255 raft_consensus.cc:2781] T d44932f8f08e4ea7acc05b8ec511f045 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:02.165163 11255 raft_consensus.cc:482] T d44932f8f08e4ea7acc05b8ec511f045 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:02.165534 11255 raft_consensus.cc:3037] T d44932f8f08e4ea7acc05b8ec511f045 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:02.152400 11049 rpcz_store.cc:276] 0924 00:21:00.397694 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:00.397828 (+ 134us) service_pool.cc:227] Handling call
0924 00:21:02.022772 (+1624944us) ts_tablet_manager.cc:1145] Deleting tablet 6ac9c7da7fb546f6ae935371799cbad6
0924 00:21:02.022830 (+ 58us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:02.074301 (+ 51471us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:02.150274 (+ 75973us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:02.152074 (+ 1800us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1513046}
I20260924 00:21:02.168016 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet ccb26e0fbc1948bc9a00e9e26c4d1139 (table default.Table_01b5209b9d5c4115b0bf6aef48b51bd8 [id=7e9e663acedf4a1fae2f04cde97b7907]) successfully deleted
I20260924 00:21:02.169855 11041 tablet_bootstrap.cc:492] T 195b033091b543b58ee08417eae2c0e6 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:02.170256 11041 ts_tablet_manager.cc:1397] T 195b033091b543b58ee08417eae2c0e6 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.095s user 0.014s sys 0.002s
I20260924 00:21:02.171380 11039 tablet_bootstrap.cc:492] T 5b0345252c3c496db2987c30adce3da7 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:02.171784 11039 ts_tablet_manager.cc:1397] T 5b0345252c3c496db2987c30adce3da7 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.065s user 0.010s sys 0.000s
I20260924 00:21:02.168259 11049 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 109) took 1283 ms. Trace:
I20260924 00:21:02.172443 10368 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet e0074b1184ec4883ab1016330d98c968 (table default.Table_63927aa1872e41da917d6b92c8acb310 [id=485c42c942b74abbb32bea36d25bc184]). Expected version 2 got 0
I20260924 00:21:02.172184 11049 rpcz_store.cc:276] 0924 00:21:00.884666 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:00.884847 (+ 181us) service_pool.cc:227] Handling call
0924 00:21:02.167372 (+1282525us) ts_tablet_manager.cc:1145] Deleting tablet 95a7a1d2147c4d7ea63ca37a8f0ea4e6
0924 00:21:02.167412 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:02.168205 (+ 793us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1281848}
I20260924 00:21:02.173697 11041 raft_consensus.cc:348] T 195b033091b543b58ee08417eae2c0e6 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:02.174342 11041 raft_consensus.cc:374] T 195b033091b543b58ee08417eae2c0e6 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:02.173986 11039 raft_consensus.cc:348] T 5b0345252c3c496db2987c30adce3da7 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:02.174588 11041 raft_consensus.cc:729] T 195b033091b543b58ee08417eae2c0e6 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:02.174610 11039 raft_consensus.cc:374] T 5b0345252c3c496db2987c30adce3da7 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:02.174844 11039 raft_consensus.cc:729] T 5b0345252c3c496db2987c30adce3da7 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:02.175715 10371 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 1ec36d59b5114feea144bcb72794136c (table default.Table_b1726c09065e4fbaa2e6799b3ebdbc74 [id=4dbb68e0a32349ad8da2945db650584d]). Expected version 1 got 0
I20260924 00:21:02.176542 11040 tablet_bootstrap.cc:654] T 86374d72238d42c29542e649a7a0200d P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:02.176499 10371 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 96960f3061fb43e9939b470f49edf1e9 (table default.Table_da80d8b26b9b48c79a00dd08f123f4f4 [id=6251651eb5944ee086dd564d86204148]). Expected version 1 got 0
I20260924 00:21:02.175529 11049 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 110) took 1242 ms. Trace:
I20260924 00:21:02.177208 10371 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 7463fcb18e884b1dbc478ebaa19f6cb4 (table default.Table_610074e75ae343618ae2141e342529b9 [id=8e2fe68fe5da4414b7269f526b8af683]). Expected version 1 got 0
I20260924 00:21:02.177757 10371 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 85a0ad61b7cb4e11b30e84ddc4655b64 (table default.Table_67493ad72aac4bbca26a6a322a60e5cd [id=12285914811f4220822b47a2a62b01ab]). Expected version 2 got 0
I20260924 00:21:02.177541 11039 consensus_queue.cc:260] T 5b0345252c3c496db2987c30adce3da7 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:02.178360 10371 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 5b0345252c3c496db2987c30adce3da7 (table default.Table_a5d798f7819e403a96f27e98cb0af2bb [id=30a3f7c8fb9c46cf8b883eedae25d938]). Expected version 4 got 0
I20260924 00:21:02.179132 10371 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet f294831873d548e4b9201c1bbb964096 (table default.Table_de36c9f702024914965bc6b371eda68e [id=e2cdad6dfe7846ebb3c14f5611cf0f18]). Expected version 1 got 0
I20260924 00:21:02.177145 11049 rpcz_store.cc:276] 0924 00:21:00.933349 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:00.933484 (+ 135us) service_pool.cc:227] Handling call
0924 00:21:02.175194 (+1241710us) ts_tablet_manager.cc:1145] Deleting tablet 5e624be739394e6d87d6b3fef2f46302
0924 00:21:02.175232 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:02.175480 (+ 248us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1241160}
I20260924 00:21:02.180143 11049 tablet_replica.cc:330] T ae0c9b785195491987dd8fe397d704e0 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:02.180698 10367 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet f9c6a39b2e2842c59b0f996f39d73162
I20260924 00:21:02.180799 11049 raft_consensus.cc:2229] T ae0c9b785195491987dd8fe397d704e0 P b10a9082a37b46f09d581944914c1670 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:21:02.181541 11049 pending_rounds.cc:62] T ae0c9b785195491987dd8fe397d704e0 P b10a9082a37b46f09d581944914c1670: Trying to abort 1 pending ops.
I20260924 00:21:02.181725 11049 pending_rounds.cc:68] T ae0c9b785195491987dd8fe397d704e0 P b10a9082a37b46f09d581944914c1670: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332697131744657408 op_type: NO_OP noop_request { }
I20260924 00:21:02.182054 11049 raft_consensus.cc:2866] T ae0c9b785195491987dd8fe397d704e0 P b10a9082a37b46f09d581944914c1670 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260924 00:21:02.182330 11049 raft_consensus.cc:2258] T ae0c9b785195491987dd8fe397d704e0 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:02.183895 11049 ts_tablet_manager.cc:1905] T ae0c9b785195491987dd8fe397d704e0 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:02.178813 11041 consensus_queue.cc:260] T 195b033091b543b58ee08417eae2c0e6 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:02.185961 10921 tablet_service.cc:1514] Processing DeleteTablet for tablet bcd8a6b92b8e492d96994195fa7f3ae5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:58 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:02.188663 10778 tablet_service.cc:1514] Processing DeleteTablet for tablet ccb26e0fbc1948bc9a00e9e26c4d1139 with delete_type TABLET_DATA_DELETED (replaced by tablet f9c6a39b2e2842c59b0f996f39d73162) from {username='slave'} at 127.0.0.1:35996
W20260924 00:21:02.189314 10358 catalog_manager.cc:4808] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet bcd8a6b92b8e492d96994195fa7f3ae5 because the tablet was not found. No further retry: Not found: Tablet not found: bcd8a6b92b8e492d96994195fa7f3ae5
I20260924 00:21:02.189425 10777 tablet_service.cc:1514] Processing DeleteTablet for tablet d44932f8f08e4ea7acc05b8ec511f045 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:58 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:02.190333 11039 ts_tablet_manager.cc:1428] T 5b0345252c3c496db2987c30adce3da7 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.018s user 0.004s sys 0.000s
W20260924 00:21:02.190747 10361 catalog_manager.cc:4808] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet ccb26e0fbc1948bc9a00e9e26c4d1139 because the tablet was not found. No further retry: Not found: Tablet not found: ccb26e0fbc1948bc9a00e9e26c4d1139
I20260924 00:21:02.191170 11039 tablet_bootstrap.cc:492] T fcf651adb1d3459facfe3a8df2a91ed7 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:02.191504 11156 tablet_replica.cc:330] T d44932f8f08e4ea7acc05b8ec511f045 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:02.193872 11255 raft_consensus.cc:504] T d44932f8f08e4ea7acc05b8ec511f045 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:02.196476 11156 raft_consensus.cc:2229] T d44932f8f08e4ea7acc05b8ec511f045 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:02.197093 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d44932f8f08e4ea7acc05b8ec511f045" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef"
I20260924 00:21:02.197625 10942 raft_consensus.cc:3037] T d44932f8f08e4ea7acc05b8ec511f045 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:02.198417 10921 tablet_service.cc:1514] Processing DeleteTablet for tablet ccb26e0fbc1948bc9a00e9e26c4d1139 with delete_type TABLET_DATA_DELETED (replaced by tablet f9c6a39b2e2842c59b0f996f39d73162) from {username='slave'} at 127.0.0.1:35358
W20260924 00:21:02.199815 10358 catalog_manager.cc:4808] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet ccb26e0fbc1948bc9a00e9e26c4d1139 because the tablet was not found. No further retry: Not found: Tablet not found: ccb26e0fbc1948bc9a00e9e26c4d1139
I20260924 00:21:02.200209 11156 raft_consensus.cc:2258] T d44932f8f08e4ea7acc05b8ec511f045 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:02.200752 11041 ts_tablet_manager.cc:1428] T 195b033091b543b58ee08417eae2c0e6 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.030s user 0.005s sys 0.001s
I20260924 00:21:02.201098 10374 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_9b991221b38645378eda8c1403ac0ce7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:02.202762 10374 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9b991221b38645378eda8c1403ac0ce7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:02.203917 10942 raft_consensus.cc:2454] T d44932f8f08e4ea7acc05b8ec511f045 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 1.
I20260924 00:21:02.204658 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d44932f8f08e4ea7acc05b8ec511f045" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:21:02.204922 10732 leader_election.cc:302] T d44932f8f08e4ea7acc05b8ec511f045 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:02.205196 10654 raft_consensus.cc:3037] T d44932f8f08e4ea7acc05b8ec511f045 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
W20260924 00:21:02.205549 10732 raft_consensus.cc:2670] T d44932f8f08e4ea7acc05b8ec511f045 P 48fbd603abe146c585d15196e1b5be08: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260924 00:21:02.207472 10366 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7463fcb18e884b1dbc478ebaa19f6cb4 consensus state has no leader
I20260924 00:21:02.210542 10368 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet 85a0ad61b7cb4e11b30e84ddc4655b64 (table default.Table_67493ad72aac4bbca26a6a322a60e5cd [id=12285914811f4220822b47a2a62b01ab]). Expected version 2 got 0
I20260924 00:21:02.212105 11041 tablet_bootstrap.cc:492] T f77aea0aad5a430f941aaec0a1c19640 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:02.215588 11260 raft_consensus.cc:482] T 666755ec87bb496db75e43ec9204ec5b P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:02.215971 10654 raft_consensus.cc:2454] T d44932f8f08e4ea7acc05b8ec511f045 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 1.
I20260924 00:21:02.216010 11260 raft_consensus.cc:504] T 666755ec87bb496db75e43ec9204ec5b P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:02.216687 11255 leader_election.cc:288] T d44932f8f08e4ea7acc05b8ec511f045 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Requested vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:02.217896 11260 leader_election.cc:288] T 666755ec87bb496db75e43ec9204ec5b P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:21:02.218909 11156 ts_tablet_manager.cc:1905] T d44932f8f08e4ea7acc05b8ec511f045 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:02.217887 10366 catalog_manager.cc:5495] T f294831873d548e4b9201c1bbb964096 P c8626e300d0644b0908797ad61bd7cef reported cstate change: term changed from 0 to 1, leader changed from <none> to c8626e300d0644b0908797ad61bd7cef (127.10.4.67). New cstate: current_term: 1 leader_uuid: "c8626e300d0644b0908797ad61bd7cef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } health_report { overall_health: UNKNOWN } } }
I20260924 00:21:02.219820 10366 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet f294831873d548e4b9201c1bbb964096 (table default.Table_de36c9f702024914965bc6b371eda68e [id=e2cdad6dfe7846ebb3c14f5611cf0f18]). Expected version 1 got 0
I20260924 00:21:02.219942 10798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "666755ec87bb496db75e43ec9204ec5b" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:02.221143 10366 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 5b0345252c3c496db2987c30adce3da7 (table default.Table_a5d798f7819e403a96f27e98cb0af2bb [id=30a3f7c8fb9c46cf8b883eedae25d938]). Expected version 4 got 0
W20260924 00:21:02.221910 10592 leader_election.cc:341] T 666755ec87bb496db75e43ec9204ec5b P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:02.222721 11039 tablet_bootstrap.cc:654] T fcf651adb1d3459facfe3a8df2a91ed7 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:02.222643 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "666755ec87bb496db75e43ec9204ec5b" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
W20260924 00:21:02.223805 10589 leader_election.cc:341] T 666755ec87bb496db75e43ec9204ec5b P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:02.224179 10589 leader_election.cc:302] T 666755ec87bb496db75e43ec9204ec5b P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:02.225134 11260 raft_consensus.cc:2726] T 666755ec87bb496db75e43ec9204ec5b P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:02.225569 10777 tablet_service.cc:1467] Processing CreateTablet for tablet 72d3bb80e1034881b94eb75de5d900c2 (DEFAULT_TABLE table=default.Table_7b8c1651b4974bd5b74177dddadf5c12 [id=ca942906de0f4019bbdf72ec886fe6fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:02.226733 10777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72d3bb80e1034881b94eb75de5d900c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:02.239967 11040 tablet_bootstrap.cc:492] T 86374d72238d42c29542e649a7a0200d P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:02.239943 11041 tablet_bootstrap.cc:654] T f77aea0aad5a430f941aaec0a1c19640 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:02.240355 11040 ts_tablet_manager.cc:1397] T 86374d72238d42c29542e649a7a0200d P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.133s user 0.007s sys 0.001s
I20260924 00:21:02.241883 10367 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_ff0bbc1b70c2417e9c69a7f7492b2981"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:02.242496 11040 raft_consensus.cc:348] T 86374d72238d42c29542e649a7a0200d P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:02.243088 11040 raft_consensus.cc:374] T 86374d72238d42c29542e649a7a0200d P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:02.243309 11040 raft_consensus.cc:729] T 86374d72238d42c29542e649a7a0200d P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
W20260924 00:21:02.243841 10367 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ff0bbc1b70c2417e9c69a7f7492b2981 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:02.246729 11039 tablet_bootstrap.cc:492] T fcf651adb1d3459facfe3a8df2a91ed7 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:02.247128 11039 ts_tablet_manager.cc:1397] T fcf651adb1d3459facfe3a8df2a91ed7 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.056s user 0.008s sys 0.000s
W20260924 00:21:02.248839 10359 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.126:34479, user_credentials={real_user=slave}} blocked reactor thread for 85251.6us
I20260924 00:21:02.249539 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 6ac9c7da7fb546f6ae935371799cbad6 (table default.Table_b22d604fe0dc4cad9d63baa9ffa5005c [id=9aee3bf701b640cfb7bcba395d4225b4]) successfully deleted
I20260924 00:21:02.249351 11039 raft_consensus.cc:348] T fcf651adb1d3459facfe3a8df2a91ed7 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:02.249954 11039 raft_consensus.cc:374] T fcf651adb1d3459facfe3a8df2a91ed7 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:02.249692 11040 consensus_queue.cc:260] T 86374d72238d42c29542e649a7a0200d P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
W20260924 00:21:02.250093 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 95a7a1d2147c4d7ea63ca37a8f0ea4e6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 95a7a1d2147c4d7ea63ca37a8f0ea4e6 already in progress: creating tablet
I20260924 00:21:02.250181 11039 raft_consensus.cc:729] T fcf651adb1d3459facfe3a8df2a91ed7 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
W20260924 00:21:02.250639 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 5e624be739394e6d87d6b3fef2f46302 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5e624be739394e6d87d6b3fef2f46302 already in progress: creating tablet
I20260924 00:21:02.250736 11039 consensus_queue.cc:260] T fcf651adb1d3459facfe3a8df2a91ed7 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:02.258204 11049 ts_tablet_manager.cc:1918] T ae0c9b785195491987dd8fe397d704e0 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:02.258559 11049 log.cc:1196] T ae0c9b785195491987dd8fe397d704e0 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/ae0c9b785195491987dd8fe397d704e0
W20260924 00:21:02.258569 10369 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 675b9e33edcf4d4ea7fa428b476e3945 consensus state has no leader
I20260924 00:21:02.259259 11049 ts_tablet_manager.cc:1939] T ae0c9b785195491987dd8fe397d704e0 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:02.262773 11040 ts_tablet_manager.cc:1428] T 86374d72238d42c29542e649a7a0200d P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.022s user 0.003s sys 0.004s
I20260924 00:21:02.263347 10370 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_5333360c3b164ff9a536584172195b56"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:02.264791 10370 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5333360c3b164ff9a536584172195b56 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:02.267520 11041 tablet_bootstrap.cc:492] T f77aea0aad5a430f941aaec0a1c19640 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:02.267912 11041 ts_tablet_manager.cc:1397] T f77aea0aad5a430f941aaec0a1c19640 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.056s user 0.011s sys 0.000s
I20260924 00:21:02.270851 11041 raft_consensus.cc:348] T f77aea0aad5a430f941aaec0a1c19640 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:02.271513 11041 raft_consensus.cc:374] T f77aea0aad5a430f941aaec0a1c19640 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:02.271750 11041 raft_consensus.cc:729] T f77aea0aad5a430f941aaec0a1c19640 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:02.273500 11041 consensus_queue.cc:260] T f77aea0aad5a430f941aaec0a1c19640 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:02.275028 11039 ts_tablet_manager.cc:1428] T fcf651adb1d3459facfe3a8df2a91ed7 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.028s user 0.007s sys 0.000s
I20260924 00:21:02.280026 11040 tablet_bootstrap.cc:492] T 5d7d52be4f1f4fc29b8172b9fa878f76 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:02.283643 10921 tablet_service.cc:1467] Processing CreateTablet for tablet 72d3bb80e1034881b94eb75de5d900c2 (DEFAULT_TABLE table=default.Table_7b8c1651b4974bd5b74177dddadf5c12 [id=ca942906de0f4019bbdf72ec886fe6fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:02.284132 11039 tablet_bootstrap.cc:492] T 59c077eb68794340ac900dbedc7056ee P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:02.284718 10921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72d3bb80e1034881b94eb75de5d900c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:02.287379 10635 tablet_service.cc:1467] Processing CreateTablet for tablet 72d3bb80e1034881b94eb75de5d900c2 (DEFAULT_TABLE table=default.Table_7b8c1651b4974bd5b74177dddadf5c12 [id=ca942906de0f4019bbdf72ec886fe6fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:02.288334 10635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72d3bb80e1034881b94eb75de5d900c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:02.289099 11039 tablet_bootstrap.cc:654] T 59c077eb68794340ac900dbedc7056ee P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:02.292675 10778 tablet_service.cc:1467] Processing CreateTablet for tablet 4e9f00abffe14046bf47a6cae7010690 (DEFAULT_TABLE table=default.Table_6fd2c4af10ab43ffafa1f24e552df8d3 [id=423a3ef82b0f488fa8fb082d4e6f63a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:02.293936 10778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e9f00abffe14046bf47a6cae7010690. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:02.299639 11260 raft_consensus.cc:482] T 195b033091b543b58ee08417eae2c0e6 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:02.300088 11260 raft_consensus.cc:504] T 195b033091b543b58ee08417eae2c0e6 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:02.301824 11260 leader_election.cc:288] T 195b033091b543b58ee08417eae2c0e6 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:02.303499 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "195b033091b543b58ee08417eae2c0e6" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:02.305080 10798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "195b033091b543b58ee08417eae2c0e6" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:02.305402 10589 leader_election.cc:341] T 195b033091b543b58ee08417eae2c0e6 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:02.306306 11255 raft_consensus.cc:482] T 86374d72238d42c29542e649a7a0200d P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:02.306700 11255 raft_consensus.cc:504] T 86374d72238d42c29542e649a7a0200d P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:02.308423 11255 leader_election.cc:288] T 86374d72238d42c29542e649a7a0200d P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
W20260924 00:21:02.309747 10592 leader_election.cc:341] T 195b033091b543b58ee08417eae2c0e6 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:02.310096 10592 leader_election.cc:302] T 195b033091b543b58ee08417eae2c0e6 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:02.310214 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86374d72238d42c29542e649a7a0200d" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:02.310752 11260 raft_consensus.cc:2726] T 195b033091b543b58ee08417eae2c0e6 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:02.310792 10942 raft_consensus.cc:2454] T 86374d72238d42c29542e649a7a0200d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 0.
I20260924 00:21:02.311723 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86374d72238d42c29542e649a7a0200d" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:02.312347 10654 raft_consensus.cc:2454] T 86374d72238d42c29542e649a7a0200d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 0.
I20260924 00:21:02.313488 10733 leader_election.cc:302] T 86374d72238d42c29542e649a7a0200d P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670; no voters:
I20260924 00:21:02.314349 11255 raft_consensus.cc:2781] T 86374d72238d42c29542e649a7a0200d P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:02.309911 11041 ts_tablet_manager.cc:1428] T f77aea0aad5a430f941aaec0a1c19640 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.042s user 0.004s sys 0.000s
I20260924 00:21:02.316327 10915 tablet_service.cc:1467] Processing CreateTablet for tablet 4e9f00abffe14046bf47a6cae7010690 (DEFAULT_TABLE table=default.Table_6fd2c4af10ab43ffafa1f24e552df8d3 [id=423a3ef82b0f488fa8fb082d4e6f63a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:02.317412 10915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e9f00abffe14046bf47a6cae7010690. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:02.314680 11255 raft_consensus.cc:482] T 86374d72238d42c29542e649a7a0200d P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:02.321202 11255 raft_consensus.cc:3037] T 86374d72238d42c29542e649a7a0200d P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:02.322926 10373 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 0c583d9ba1414f8ea2e605b4b6b076f8
I20260924 00:21:02.328055 10774 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c583d9ba1414f8ea2e605b4b6b076f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:01 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:02.329180 10629 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c583d9ba1414f8ea2e605b4b6b076f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:01 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:02.329856 10922 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c583d9ba1414f8ea2e605b4b6b076f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:01 UTC) from {username='slave'} at 127.0.0.1:35358
W20260924 00:21:02.332351 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 0c583d9ba1414f8ea2e605b4b6b076f8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0c583d9ba1414f8ea2e605b4b6b076f8 already in progress: creating tablet
I20260924 00:21:02.334233 11039 tablet_bootstrap.cc:492] T 59c077eb68794340ac900dbedc7056ee P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:02.334707 11039 ts_tablet_manager.cc:1397] T 59c077eb68794340ac900dbedc7056ee P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.051s user 0.009s sys 0.000s
I20260924 00:21:02.337373 11039 raft_consensus.cc:348] T 59c077eb68794340ac900dbedc7056ee P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:02.338037 11039 raft_consensus.cc:374] T 59c077eb68794340ac900dbedc7056ee P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:02.338266 11039 raft_consensus.cc:729] T 59c077eb68794340ac900dbedc7056ee P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:02.338829 11039 consensus_queue.cc:260] T 59c077eb68794340ac900dbedc7056ee P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:02.339243 11041 tablet_bootstrap.cc:492] T 772fb3d6180e499daf94ceeb9b26048d P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:02.339564 11255 raft_consensus.cc:504] T 86374d72238d42c29542e649a7a0200d P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:02.343468 11040 tablet_bootstrap.cc:654] T 5d7d52be4f1f4fc29b8172b9fa878f76 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:02.346853 11049 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 111) took 1353 ms. Trace:
I20260924 00:21:02.347165 11049 rpcz_store.cc:276] 0924 00:21:00.992484 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:00.992628 (+ 144us) service_pool.cc:227] Handling call
0924 00:21:02.180004 (+1187376us) ts_tablet_manager.cc:1145] Deleting tablet ae0c9b785195491987dd8fe397d704e0
0924 00:21:02.180041 (+ 37us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:02.234278 (+ 54237us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:02.257992 (+ 23714us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:02.346004 (+ 88012us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1186789}
I20260924 00:21:02.348362 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet ae0c9b785195491987dd8fe397d704e0 (table default.Table_d61af65b53ce4871b7b24752281af745 [id=0edc6c93a1ba4c8d81755129301c83de]) successfully deleted
I20260924 00:21:02.350689 11041 tablet_bootstrap.cc:654] T 772fb3d6180e499daf94ceeb9b26048d P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:02.354735 11039 ts_tablet_manager.cc:1428] T 59c077eb68794340ac900dbedc7056ee P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.020s user 0.004s sys 0.004s
I20260924 00:21:02.355598 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86374d72238d42c29542e649a7a0200d" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef"
I20260924 00:21:02.356141 10942 raft_consensus.cc:3037] T 86374d72238d42c29542e649a7a0200d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:02.363373 10635 tablet_service.cc:1467] Processing CreateTablet for tablet 4e9f00abffe14046bf47a6cae7010690 (DEFAULT_TABLE table=default.Table_6fd2c4af10ab43ffafa1f24e552df8d3 [id=423a3ef82b0f488fa8fb082d4e6f63a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:02.364598 10942 raft_consensus.cc:2454] T 86374d72238d42c29542e649a7a0200d P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 1.
I20260924 00:21:02.364837 10635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e9f00abffe14046bf47a6cae7010690. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:02.365788 10732 leader_election.cc:302] T 86374d72238d42c29542e649a7a0200d P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:02.371415 11041 tablet_bootstrap.cc:492] T 772fb3d6180e499daf94ceeb9b26048d P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:02.371850 11041 ts_tablet_manager.cc:1397] T 772fb3d6180e499daf94ceeb9b26048d P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.033s user 0.008s sys 0.002s
I20260924 00:21:02.372130 11039 tablet_bootstrap.cc:492] T 7463fcb18e884b1dbc478ebaa19f6cb4 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:02.373426 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86374d72238d42c29542e649a7a0200d" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:21:02.373980 10654 raft_consensus.cc:3037] T 86374d72238d42c29542e649a7a0200d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
W20260924 00:21:02.375121 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 4420cead1f284af8958f189557833098 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4420cead1f284af8958f189557833098 already in progress: creating tablet
I20260924 00:21:02.375761 11255 leader_election.cc:288] T 86374d72238d42c29542e649a7a0200d P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Requested vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:02.376248 11255 raft_consensus.cc:2781] T 86374d72238d42c29542e649a7a0200d P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:02.376371 10774 tablet_service.cc:1467] Processing CreateTablet for tablet cb51d5c6460049108cd7e1974821e578 (DEFAULT_TABLE table=default.Table_0c36f6e4aa264a4394fd0bc65fd035a8 [id=ed975cbe1bb04e3194882bed0f4ec0b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:02.376639 11255 raft_consensus.cc:686] T 86374d72238d42c29542e649a7a0200d P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: Becoming Leader. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Running, Role: LEADER
I20260924 00:21:02.376878 11039 tablet_bootstrap.cc:654] T 7463fcb18e884b1dbc478ebaa19f6cb4 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:02.377434 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb51d5c6460049108cd7e1974821e578. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:02.377408 11255 consensus_queue.cc:237] T 86374d72238d42c29542e649a7a0200d P 48fbd603abe146c585d15196e1b5be08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:02.380002 11049 tablet_replica.cc:330] T 666755ec87bb496db75e43ec9204ec5b P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:02.381616 10654 raft_consensus.cc:2454] T 86374d72238d42c29542e649a7a0200d P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 1.
I20260924 00:21:02.373940 11041 raft_consensus.cc:348] T 772fb3d6180e499daf94ceeb9b26048d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:02.385066 11156 ts_tablet_manager.cc:1918] T d44932f8f08e4ea7acc05b8ec511f045 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:02.385429 11156 log.cc:1196] T d44932f8f08e4ea7acc05b8ec511f045 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/d44932f8f08e4ea7acc05b8ec511f045
I20260924 00:21:02.386018 11156 ts_tablet_manager.cc:1939] T d44932f8f08e4ea7acc05b8ec511f045 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:02.385468 11041 raft_consensus.cc:374] T 772fb3d6180e499daf94ceeb9b26048d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:02.386404 11041 raft_consensus.cc:729] T 772fb3d6180e499daf94ceeb9b26048d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:02.386678 11049 raft_consensus.cc:2229] T 666755ec87bb496db75e43ec9204ec5b P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:02.389766 11049 raft_consensus.cc:2258] T 666755ec87bb496db75e43ec9204ec5b P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:02.389448 11041 consensus_queue.cc:260] T 772fb3d6180e499daf94ceeb9b26048d P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:02.398828 11041 ts_tablet_manager.cc:1428] T 772fb3d6180e499daf94ceeb9b26048d P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.027s user 0.005s sys 0.001s
I20260924 00:21:02.402300 11040 tablet_bootstrap.cc:492] T 5d7d52be4f1f4fc29b8172b9fa878f76 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:02.402700 11040 ts_tablet_manager.cc:1397] T 5d7d52be4f1f4fc29b8172b9fa878f76 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.123s user 0.008s sys 0.000s
I20260924 00:21:02.400043 11260 consensus_queue.cc:1035] T 85a0ad61b7cb4e11b30e84ddc4655b64 P b10a9082a37b46f09d581944914c1670 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:02.402861 10373 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_d01ccf505bd94805b62c8f0755dad1d1" } modify_external_catalogs: true
I20260924 00:21:02.403445 10373 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_d01ccf505bd94805b62c8f0755dad1d1" } modify_external_catalogs: true
I20260924 00:21:02.408830 11041 tablet_bootstrap.cc:492] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:02.410722 11254 raft_consensus.cc:482] T 5d7d52be4f1f4fc29b8172b9fa878f76 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:02.411158 11254 raft_consensus.cc:504] T 5d7d52be4f1f4fc29b8172b9fa878f76 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:02.412477 11260 raft_consensus.cc:482] T f77aea0aad5a430f941aaec0a1c19640 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:02.413021 11254 leader_election.cc:288] T 5d7d52be4f1f4fc29b8172b9fa878f76 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:02.412925 10372 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_e94c810f23d34f62bfa2bb1a7e5f0a02" } new_table_name: "default.Table_dca718fc1c1a4e518bbc52483fdd7a09" modify_external_catalogs: true
I20260924 00:21:02.391705 11049 ts_tablet_manager.cc:1905] T 666755ec87bb496db75e43ec9204ec5b P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:02.412878 11260 raft_consensus.cc:504] T f77aea0aad5a430f941aaec0a1c19640 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:02.414662 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d7d52be4f1f4fc29b8172b9fa878f76" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:02.415280 10654 raft_consensus.cc:2454] T 5d7d52be4f1f4fc29b8172b9fa878f76 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
I20260924 00:21:02.415427 11260 leader_election.cc:288] T f77aea0aad5a430f941aaec0a1c19640 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:02.416415 10876 leader_election.cc:302] T 5d7d52be4f1f4fc29b8172b9fa878f76 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:02.417243 11254 raft_consensus.cc:2781] T 5d7d52be4f1f4fc29b8172b9fa878f76 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:02.417179 10796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f77aea0aad5a430f941aaec0a1c19640" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:02.417547 11254 raft_consensus.cc:482] T 5d7d52be4f1f4fc29b8172b9fa878f76 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:02.417820 11254 raft_consensus.cc:3037] T 5d7d52be4f1f4fc29b8172b9fa878f76 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:02.418602 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d7d52be4f1f4fc29b8172b9fa878f76" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:02.425277 11039 tablet_bootstrap.cc:492] T 7463fcb18e884b1dbc478ebaa19f6cb4 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:02.425753 11039 ts_tablet_manager.cc:1397] T 7463fcb18e884b1dbc478ebaa19f6cb4 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.054s user 0.012s sys 0.002s
I20260924 00:21:02.428493 11039 raft_consensus.cc:348] T 7463fcb18e884b1dbc478ebaa19f6cb4 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:02.429869 10365 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet f3432ac96da94d5099bcf9764c2b32e1
I20260924 00:21:02.431193 10365 catalog_manager.cc:6565] ReplaceTablet: tablet f3432ac96da94d5099bcf9764c2b32e1 deleted and replaced by tablet 5b43ce1fed0749b8b076b9b026914e07
I20260924 00:21:02.434233 10634 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d7d52be4f1f4fc29b8172b9fa878f76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:57 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:02.434830 11039 raft_consensus.cc:374] T 7463fcb18e884b1dbc478ebaa19f6cb4 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:02.434902 11041 tablet_bootstrap.cc:654] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:02.435202 11039 raft_consensus.cc:729] T 7463fcb18e884b1dbc478ebaa19f6cb4 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:02.435089 10629 tablet_service.cc:1467] Processing CreateTablet for tablet cb51d5c6460049108cd7e1974821e578 (DEFAULT_TABLE table=default.Table_0c36f6e4aa264a4394fd0bc65fd035a8 [id=ed975cbe1bb04e3194882bed0f4ec0b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:02.435408 10634 tablet_service.cc:1514] Processing DeleteTablet for tablet ae0c9b785195491987dd8fe397d704e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:00 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:02.436046 10636 tablet_service.cc:1514] Processing DeleteTablet for tablet f3432ac96da94d5099bcf9764c2b32e1 with delete_type TABLET_DATA_DELETED (replaced by tablet 5b43ce1fed0749b8b076b9b026914e07) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:02.433741 11254 raft_consensus.cc:504] T 5d7d52be4f1f4fc29b8172b9fa878f76 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:02.439610 10922 tablet_service.cc:1514] Processing DeleteTablet for tablet f3432ac96da94d5099bcf9764c2b32e1 with delete_type TABLET_DATA_DELETED (replaced by tablet 5b43ce1fed0749b8b076b9b026914e07) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:02.439796 10636 tablet_service.cc:1514] Processing DeleteTablet for tablet 86374d72238d42c29542e649a7a0200d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:58 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:02.440241 11039 consensus_queue.cc:260] T 7463fcb18e884b1dbc478ebaa19f6cb4 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
W20260924 00:21:02.443637 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet f3432ac96da94d5099bcf9764c2b32e1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f3432ac96da94d5099bcf9764c2b32e1 already in progress: creating tablet
I20260924 00:21:02.444216 10922 tablet_service.cc:1467] Processing CreateTablet for tablet b36cbb6520de4df58b88cfa53c20ee7a (DEFAULT_TABLE table=default.Table_dae0ffc8534047588f6f7cb13a610af5 [id=138be013fbcb4f01a6358d062200fe8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:02.445266 10922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b36cbb6520de4df58b88cfa53c20ee7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:02.446879 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d7d52be4f1f4fc29b8172b9fa878f76" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:21:02.447425 10654 raft_consensus.cc:3037] T 5d7d52be4f1f4fc29b8172b9fa878f76 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:02.451225 10629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb51d5c6460049108cd7e1974821e578. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:21:02.453903 10592 leader_election.cc:341] T f77aea0aad5a430f941aaec0a1c19640 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:02.454576 10654 raft_consensus.cc:2454] T 5d7d52be4f1f4fc29b8172b9fa878f76 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 1.
I20260924 00:21:02.454749 10636 tablet_service.cc:1514] Processing DeleteTablet for tablet d44932f8f08e4ea7acc05b8ec511f045 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:58 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:02.455677 10876 leader_election.cc:302] T 5d7d52be4f1f4fc29b8172b9fa878f76 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:02.460659 11256 raft_consensus.cc:2781] T 5d7d52be4f1f4fc29b8172b9fa878f76 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:02.461138 11256 raft_consensus.cc:686] T 5d7d52be4f1f4fc29b8172b9fa878f76 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Becoming Leader. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Running, Role: LEADER
I20260924 00:21:02.461861 11256 consensus_queue.cc:237] T 5d7d52be4f1f4fc29b8172b9fa878f76 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:02.464535 10636 tablet_service.cc:1467] Processing CreateTablet for tablet b36cbb6520de4df58b88cfa53c20ee7a (DEFAULT_TABLE table=default.Table_dae0ffc8534047588f6f7cb13a610af5 [id=138be013fbcb4f01a6358d062200fe8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:02.465512 10636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b36cbb6520de4df58b88cfa53c20ee7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:02.468393 11049 ts_tablet_manager.cc:1918] T 666755ec87bb496db75e43ec9204ec5b P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:02.468739 11049 log.cc:1196] T 666755ec87bb496db75e43ec9204ec5b P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/666755ec87bb496db75e43ec9204ec5b
I20260924 00:21:02.469376 11049 ts_tablet_manager.cc:1939] T 666755ec87bb496db75e43ec9204ec5b P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:02.470535 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 666755ec87bb496db75e43ec9204ec5b (table default.Table_1347839140d544c3b4aee063e2efe906 [id=00b8c1d674cc460bb7b2425c55037246]) successfully deleted
I20260924 00:21:02.472085 11039 ts_tablet_manager.cc:1428] T 7463fcb18e884b1dbc478ebaa19f6cb4 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.046s user 0.005s sys 0.000s
I20260924 00:21:02.472426 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f77aea0aad5a430f941aaec0a1c19640" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:02.472642 11254 leader_election.cc:288] T 5d7d52be4f1f4fc29b8172b9fa878f76 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
W20260924 00:21:02.474820 10589 leader_election.cc:341] T f77aea0aad5a430f941aaec0a1c19640 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:02.475211 10589 leader_election.cc:302] T f77aea0aad5a430f941aaec0a1c19640 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
W20260924 00:21:02.476085 10330 debug-util.cc:398] Leaking SignalData structure 0x7b08000b27e0 after lost signal to thread 10267
I20260924 00:21:02.476439 11260 raft_consensus.cc:2726] T f77aea0aad5a430f941aaec0a1c19640 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260924 00:21:02.477021 10330 debug-util.cc:398] Leaking SignalData structure 0x7b08000c0860 after lost signal to thread 10333
I20260924 00:21:02.480198 10369 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_a01f8ccbfb7a4cff9315ec54f8f2f50e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:02.480690 10796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d7d52be4f1f4fc29b8172b9fa878f76" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:21:02.481485 11049 tablet_replica.cc:330] T ccb26e0fbc1948bc9a00e9e26c4d1139 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
W20260924 00:21:02.481673 10369 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a01f8ccbfb7a4cff9315ec54f8f2f50e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:02.482177 11049 raft_consensus.cc:2229] T ccb26e0fbc1948bc9a00e9e26c4d1139 P b10a9082a37b46f09d581944914c1670 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:21:02.482975 11049 raft_consensus.cc:2258] T ccb26e0fbc1948bc9a00e9e26c4d1139 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:02.487643 10365 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_95d86f2882084cb6821c2aa52497b554" } new_table_name: "default.Table_0cc93ce7fcfb44089209422666978c62" modify_external_catalogs: true
I20260924 00:21:02.489158 10921 tablet_service.cc:1467] Processing CreateTablet for tablet cb51d5c6460049108cd7e1974821e578 (DEFAULT_TABLE table=default.Table_0c36f6e4aa264a4394fd0bc65fd035a8 [id=ed975cbe1bb04e3194882bed0f4ec0b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:02.489909 11041 tablet_bootstrap.cc:492] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:02.490306 11041 ts_tablet_manager.cc:1397] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.082s user 0.009s sys 0.000s
I20260924 00:21:02.492312 11041 raft_consensus.cc:348] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:02.492936 11041 raft_consensus.cc:374] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:02.493220 11041 raft_consensus.cc:729] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:02.493757 11041 consensus_queue.cc:260] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:02.495973 11041 ts_tablet_manager.cc:1428] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260924 00:21:02.500120 10920 tablet_service.cc:1467] Processing CreateTablet for tablet 59a0a950a8144e99b2ed47be9a567d9a (DEFAULT_TABLE table=default.Table_faaa8b08d77d4384a41b2008c9cc98ea [id=3fce8b5a6e82492b8a2355183e566142]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:02.500602 10634 tablet_service.cc:1467] Processing CreateTablet for tablet 59a0a950a8144e99b2ed47be9a567d9a (DEFAULT_TABLE table=default.Table_faaa8b08d77d4384a41b2008c9cc98ea [id=3fce8b5a6e82492b8a2355183e566142]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:02.501588 10634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59a0a950a8144e99b2ed47be9a567d9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:02.503364 11041 tablet_bootstrap.cc:492] T 119d83562ef546cab496cd7b44fecc15 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
W20260924 00:21:02.505015 10371 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.4.65:55561 (request call id 43) took 936 ms (client timeout 1000 ms). Trace:
I20260924 00:21:02.506337 10778 tablet_service.cc:1514] Processing DeleteTablet for tablet f3432ac96da94d5099bcf9764c2b32e1 with delete_type TABLET_DATA_DELETED (replaced by tablet 5b43ce1fed0749b8b076b9b026914e07) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:02.506320 10919 tablet_service.cc:1467] Processing CreateTablet for tablet 6bcfd729e9bd4040b3384ab24164e80d (DEFAULT_TABLE table=default.Table_e86eeb97d87d4f9ea7edfe269cc1d790 [id=df2a5a74db65431c9ecb750184f64709]), partition=RANGE (key) PARTITION UNBOUNDED
W20260924 00:21:02.505391 10371 rpcz_store.cc:269] 0924 00:21:01.531831 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:01.535735 (+ 3904us) service_pool.cc:227] Handling call
0924 00:21:02.468612 (+932877us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":14}
I20260924 00:21:02.510663 10630 tablet_service.cc:1467] Processing CreateTablet for tablet 6bcfd729e9bd4040b3384ab24164e80d (DEFAULT_TABLE table=default.Table_e86eeb97d87d4f9ea7edfe269cc1d790 [id=df2a5a74db65431c9ecb750184f64709]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:02.512151 10630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bcfd729e9bd4040b3384ab24164e80d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:02.519428 10919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bcfd729e9bd4040b3384ab24164e80d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:02.519579 11049 ts_tablet_manager.cc:1905] T ccb26e0fbc1948bc9a00e9e26c4d1139 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:02.485271 11301 consensus_queue.cc:1035] T 1ec36d59b5114feea144bcb72794136c P 48fbd603abe146c585d15196e1b5be08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 00:21:02.525478 10920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59a0a950a8144e99b2ed47be9a567d9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:02.529031 10374 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_b617c841f36c48be8b10898aff4bda2e" } modify_external_catalogs: true
I20260924 00:21:02.529572 11039 tablet_bootstrap.cc:492] T 666755ec87bb496db75e43ec9204ec5b P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:02.529548 10374 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_b617c841f36c48be8b10898aff4bda2e" } modify_external_catalogs: true
I20260924 00:21:02.542704 10368 catalog_manager.cc:6040] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: default.Table_63927aa1872e41da917d6b92c8acb310 [id=485c42c942b74abbb32bea36d25bc184] alter complete (version 2)
I20260924 00:21:02.545132 11041 tablet_bootstrap.cc:654] T 119d83562ef546cab496cd7b44fecc15 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:02.545311 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet d44932f8f08e4ea7acc05b8ec511f045 (table default.Table_321e6959ada84ccea6413114309bc3b8 [id=8092ca5df917480aba7e26a4cedf9fbd]) successfully deleted
W20260924 00:21:02.547487 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 0c583d9ba1414f8ea2e605b4b6b076f8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0c583d9ba1414f8ea2e605b4b6b076f8 already in progress: creating tablet
I20260924 00:21:02.547823 10778 tablet_service.cc:1467] Processing CreateTablet for tablet 59a0a950a8144e99b2ed47be9a567d9a (DEFAULT_TABLE table=default.Table_faaa8b08d77d4384a41b2008c9cc98ea [id=3fce8b5a6e82492b8a2355183e566142]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:02.548929 10778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59a0a950a8144e99b2ed47be9a567d9a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:21:02.551294 10360 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 675b9e33edcf4d4ea7fa428b476e3945 (table default.Table_ee73693b7a6e46a3af53f7828da00f9e [id=9359b05533064c38b9bd589c0bb06471], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 675b9e33edcf4d4ea7fa428b476e3945 consensus state has no leader
W20260924 00:21:02.555485 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet f3432ac96da94d5099bcf9764c2b32e1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f3432ac96da94d5099bcf9764c2b32e1 already in progress: creating tablet
I20260924 00:21:02.559005 10773 tablet_service.cc:1467] Processing CreateTablet for tablet 10aa64ed664f40629f5f134052df8f5f (DEFAULT_TABLE table=default.Table_d56dcf7ebba04b769e2e233cc7d9fa0e [id=389f9c4d091d4c1da824aafd4f9ebb2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:02.560039 10773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10aa64ed664f40629f5f134052df8f5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:02.561833 10777 tablet_service.cc:1467] Processing CreateTablet for tablet 6bcfd729e9bd4040b3384ab24164e80d (DEFAULT_TABLE table=default.Table_e86eeb97d87d4f9ea7edfe269cc1d790 [id=df2a5a74db65431c9ecb750184f64709]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:02.562561 11260 consensus_queue.cc:1035] T 96960f3061fb43e9939b470f49edf1e9 P b10a9082a37b46f09d581944914c1670 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:02.563007 10777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bcfd729e9bd4040b3384ab24164e80d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:02.565302 10921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb51d5c6460049108cd7e1974821e578. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:02.568128 10774 tablet_service.cc:1467] Processing CreateTablet for tablet b36cbb6520de4df58b88cfa53c20ee7a (DEFAULT_TABLE table=default.Table_dae0ffc8534047588f6f7cb13a610af5 [id=138be013fbcb4f01a6358d062200fe8f]), partition=RANGE (key) PARTITION UNBOUNDED
W20260924 00:21:02.569865 10878 leader_election.cc:341] T 5d7d52be4f1f4fc29b8172b9fa878f76 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:02.578334 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b36cbb6520de4df58b88cfa53c20ee7a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:21:02.591507 10878 leader_election.cc:341] T 5d7d52be4f1f4fc29b8172b9fa878f76 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:02.605825 10917 tablet_service.cc:1514] Processing DeleteTablet for tablet 86374d72238d42c29542e649a7a0200d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:58 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:02.606654 11051 tablet_replica.cc:330] T 86374d72238d42c29542e649a7a0200d P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:02.606714 10366 master_service.cc:946] ReplaceTablet: received request to replace tablet 95a7a1d2147c4d7ea63ca37a8f0ea4e6 from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:02.607357 11051 raft_consensus.cc:2229] T 86374d72238d42c29542e649a7a0200d P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:02.607764 11051 raft_consensus.cc:2258] T 86374d72238d42c29542e649a7a0200d P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:02.609457 11256 raft_consensus.cc:482] T 7463fcb18e884b1dbc478ebaa19f6cb4 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:02.609894 11256 raft_consensus.cc:504] T 7463fcb18e884b1dbc478ebaa19f6cb4 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:02.612078 10635 tablet_service.cc:1467] Processing CreateTablet for tablet 3b8d74761b0b4fd0a22021d3870ec05c (DEFAULT_TABLE table=default.Table_95d86f2882084cb6821c2aa52497b554 [id=c0cd199f74494deda9d088345c4c90a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:02.612308 11256 leader_election.cc:288] T 7463fcb18e884b1dbc478ebaa19f6cb4 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:21:02.613345 11051 ts_tablet_manager.cc:1905] T 86374d72238d42c29542e649a7a0200d P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:02.615231 10368 catalog_manager.cc:5495] T 85a0ad61b7cb4e11b30e84ddc4655b64 P b10a9082a37b46f09d581944914c1670 reported cstate change: term changed from 0 to 1, leader changed from <none> to b10a9082a37b46f09d581944914c1670 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "b10a9082a37b46f09d581944914c1670" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } health_report { overall_health: HEALTHY } } }
I20260924 00:21:02.616568 10368 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 85a0ad61b7cb4e11b30e84ddc4655b64 (table default.Table_67493ad72aac4bbca26a6a322a60e5cd [id=12285914811f4220822b47a2a62b01ab]). Expected version 2 got 0
I20260924 00:21:02.617280 10368 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 195b033091b543b58ee08417eae2c0e6 (table default.Table_ca083375f0df4255b8654ed88040dfcd [id=f79ca8d11aa04d5f8f72d58b798ed4ea]). Expected version 1 got 0
I20260924 00:21:02.619197 10634 tablet_service.cc:1467] Processing CreateTablet for tablet 10aa64ed664f40629f5f134052df8f5f (DEFAULT_TABLE table=default.Table_d56dcf7ebba04b769e2e233cc7d9fa0e [id=389f9c4d091d4c1da824aafd4f9ebb2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:02.620199 10917 tablet_service.cc:1467] Processing CreateTablet for tablet 10aa64ed664f40629f5f134052df8f5f (DEFAULT_TABLE table=default.Table_d56dcf7ebba04b769e2e233cc7d9fa0e [id=389f9c4d091d4c1da824aafd4f9ebb2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:02.621347 10917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10aa64ed664f40629f5f134052df8f5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:02.628669 10634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10aa64ed664f40629f5f134052df8f5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:02.631889 10918 tablet_service.cc:1467] Processing CreateTablet for tablet 3b8d74761b0b4fd0a22021d3870ec05c (DEFAULT_TABLE table=default.Table_95d86f2882084cb6821c2aa52497b554 [id=c0cd199f74494deda9d088345c4c90a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:02.632881 10918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b8d74761b0b4fd0a22021d3870ec05c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:02.618160 10368 catalog_manager.cc:5495] T 96960f3061fb43e9939b470f49edf1e9 P b10a9082a37b46f09d581944914c1670 reported cstate change: term changed from 0 to 1, leader changed from <none> to b10a9082a37b46f09d581944914c1670 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "b10a9082a37b46f09d581944914c1670" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } health_report { overall_health: HEALTHY } } }
I20260924 00:21:02.633770 10368 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 96960f3061fb43e9939b470f49edf1e9 (table default.Table_da80d8b26b9b48c79a00dd08f123f4f4 [id=6251651eb5944ee086dd564d86204148]). Expected version 1 got 0
I20260924 00:21:02.635953 10635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b8d74761b0b4fd0a22021d3870ec05c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:02.643867 11040 raft_consensus.cc:348] T 5d7d52be4f1f4fc29b8172b9fa878f76 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:02.652922 11306 raft_consensus.cc:482] T 772fb3d6180e499daf94ceeb9b26048d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:02.653491 11306 raft_consensus.cc:504] T 772fb3d6180e499daf94ceeb9b26048d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:02.655571 11306 leader_election.cc:288] T 772fb3d6180e499daf94ceeb9b26048d P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:02.658934 10920 tablet_service.cc:1514] Processing DeleteTablet for tablet d44932f8f08e4ea7acc05b8ec511f045 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:58 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:02.668673 11040 raft_consensus.cc:374] T 5d7d52be4f1f4fc29b8172b9fa878f76 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:02.669453 10776 tablet_service.cc:1467] Processing CreateTablet for tablet 3b8d74761b0b4fd0a22021d3870ec05c (DEFAULT_TABLE table=default.Table_95d86f2882084cb6821c2aa52497b554 [id=c0cd199f74494deda9d088345c4c90a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:02.669795 11040 raft_consensus.cc:729] T 5d7d52be4f1f4fc29b8172b9fa878f76 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:02.670583 10776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b8d74761b0b4fd0a22021d3870ec05c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:02.671809 11040 consensus_queue.cc:260] T 5d7d52be4f1f4fc29b8172b9fa878f76 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:02.688695 10370 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_e724566b223445b7b3988eeb3e811bcd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:02.690009 10920 tablet_service.cc:1467] Processing CreateTablet for tablet e937cee22326474598cef5c1d09b59c2 (DEFAULT_TABLE table=default.Table_1ebdc131b4ce4446b021aca0fd94ae99 [id=54d197b284b34dc0a5eea440b0b1e5de]), partition=RANGE (key) PARTITION UNBOUNDED
W20260924 00:21:02.690145 10370 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e724566b223445b7b3988eeb3e811bcd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:02.691100 10920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e937cee22326474598cef5c1d09b59c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:02.696497 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "772fb3d6180e499daf94ceeb9b26048d" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:02.701012 11039 tablet_bootstrap.cc:654] T 666755ec87bb496db75e43ec9204ec5b P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:02.703806 10919 tablet_service.cc:1467] Processing CreateTablet for tablet 675b9e33edcf4d4ea7fa428b476e3945 (DEFAULT_TABLE table=default.Table_ee73693b7a6e46a3af53f7828da00f9e [id=9359b05533064c38b9bd589c0bb06471]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:02.705003 10919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 675b9e33edcf4d4ea7fa428b476e3945. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:02.708295 10373 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 7463fcb18e884b1dbc478ebaa19f6cb4
I20260924 00:21:02.711144 11040 ts_tablet_manager.cc:1428] T 5d7d52be4f1f4fc29b8172b9fa878f76 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.308s user 0.007s sys 0.003s
I20260924 00:21:02.717859 11256 raft_consensus.cc:482] T fcf651adb1d3459facfe3a8df2a91ed7 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:02.718396 11256 raft_consensus.cc:504] T fcf651adb1d3459facfe3a8df2a91ed7 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:02.721789 11256 leader_election.cc:288] T fcf651adb1d3459facfe3a8df2a91ed7 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:21:02.723658 10916 tablet_service.cc:1514] Processing DeleteTablet for tablet 7463fcb18e884b1dbc478ebaa19f6cb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:02 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:02.656720 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "772fb3d6180e499daf94ceeb9b26048d" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:02.708328 11255 consensus_queue.cc:1035] T 1ec36d59b5114feea144bcb72794136c P 48fbd603abe146c585d15196e1b5be08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:02.687274 10915 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d7d52be4f1f4fc29b8172b9fa878f76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:57 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:02.667698 10656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7463fcb18e884b1dbc478ebaa19f6cb4" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:02.737617 10656 raft_consensus.cc:2454] T 7463fcb18e884b1dbc478ebaa19f6cb4 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
I20260924 00:21:02.673695 10636 tablet_service.cc:1467] Processing CreateTablet for tablet e937cee22326474598cef5c1d09b59c2 (DEFAULT_TABLE table=default.Table_1ebdc131b4ce4446b021aca0fd94ae99 [id=54d197b284b34dc0a5eea440b0b1e5de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:02.739276 10636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e937cee22326474598cef5c1d09b59c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:02.741639 11040 tablet_bootstrap.cc:492] T f294831873d548e4b9201c1bbb964096 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
W20260924 00:21:02.745692 10372 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4c2a729a802e4abe9ab4f989a40198be has no consensus state
I20260924 00:21:02.680481 10629 tablet_service.cc:1467] Processing CreateTablet for tablet 675b9e33edcf4d4ea7fa428b476e3945 (DEFAULT_TABLE table=default.Table_ee73693b7a6e46a3af53f7828da00f9e [id=9359b05533064c38b9bd589c0bb06471]), partition=RANGE (key) PARTITION UNBOUNDED
W20260924 00:21:02.697893 10589 leader_election.cc:341] T 772fb3d6180e499daf94ceeb9b26048d P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
W20260924 00:21:02.747391 10589 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:41301, user_credentials={real_user=slave}} blocked reactor thread for 49542.9us
I20260924 00:21:02.714944 10630 tablet_service.cc:1514] Processing DeleteTablet for tablet 7463fcb18e884b1dbc478ebaa19f6cb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:02 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:02.749683 10876 leader_election.cc:302] T 7463fcb18e884b1dbc478ebaa19f6cb4 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:02.750667 11254 raft_consensus.cc:2781] T 7463fcb18e884b1dbc478ebaa19f6cb4 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:02.750985 11254 raft_consensus.cc:482] T 7463fcb18e884b1dbc478ebaa19f6cb4 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:02.751268 11254 raft_consensus.cc:3037] T 7463fcb18e884b1dbc478ebaa19f6cb4 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:02.760210 11049 ts_tablet_manager.cc:1918] T ccb26e0fbc1948bc9a00e9e26c4d1139 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:02.760587 11049 log.cc:1196] T ccb26e0fbc1948bc9a00e9e26c4d1139 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/ccb26e0fbc1948bc9a00e9e26c4d1139
I20260924 00:21:02.760098 11254 raft_consensus.cc:504] T 7463fcb18e884b1dbc478ebaa19f6cb4 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:02.761271 11049 ts_tablet_manager.cc:1939] T ccb26e0fbc1948bc9a00e9e26c4d1139 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:02.762732 11049 tablet_replica.cc:330] T 5d7d52be4f1f4fc29b8172b9fa878f76 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:02.762852 11254 leader_election.cc:288] T 7463fcb18e884b1dbc478ebaa19f6cb4 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
W20260924 00:21:02.763243 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 0c583d9ba1414f8ea2e605b4b6b076f8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0c583d9ba1414f8ea2e605b4b6b076f8 already in progress: creating tablet
I20260924 00:21:02.763418 11049 raft_consensus.cc:2229] T 5d7d52be4f1f4fc29b8172b9fa878f76 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:02.763806 11049 raft_consensus.cc:2258] T 5d7d52be4f1f4fc29b8172b9fa878f76 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 00:21:02.763852 10365 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3b8d74761b0b4fd0a22021d3870ec05c consensus state has no leader
I20260924 00:21:02.766530 11260 consensus_queue.cc:1035] T 85a0ad61b7cb4e11b30e84ddc4655b64 P b10a9082a37b46f09d581944914c1670 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:02.767690 10656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7463fcb18e884b1dbc478ebaa19f6cb4" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:21:02.768214 10656 raft_consensus.cc:3037] T 7463fcb18e884b1dbc478ebaa19f6cb4 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:02.776381 10656 raft_consensus.cc:2454] T 7463fcb18e884b1dbc478ebaa19f6cb4 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 1.
I20260924 00:21:02.777282 10629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 675b9e33edcf4d4ea7fa428b476e3945. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:02.780135 10374 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet fdc2e072c5594d0482ddc7e0716f6565
I20260924 00:21:02.781853 10876 leader_election.cc:302] T 7463fcb18e884b1dbc478ebaa19f6cb4 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:02.782583 11256 raft_consensus.cc:2781] T 7463fcb18e884b1dbc478ebaa19f6cb4 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:02.784078 11256 raft_consensus.cc:686] T 7463fcb18e884b1dbc478ebaa19f6cb4 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Becoming Leader. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Running, Role: LEADER
I20260924 00:21:02.784860 11256 consensus_queue.cc:237] T 7463fcb18e884b1dbc478ebaa19f6cb4 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:02.786412 10374 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 1ec36d59b5114feea144bcb72794136c (table default.Table_b1726c09065e4fbaa2e6799b3ebdbc74 [id=4dbb68e0a32349ad8da2945db650584d]). Expected version 1 got 0
I20260924 00:21:02.737303 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcf651adb1d3459facfe3a8df2a91ed7" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:02.789300 10654 raft_consensus.cc:2454] T fcf651adb1d3459facfe3a8df2a91ed7 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
I20260924 00:21:02.790400 10876 leader_election.cc:302] T fcf651adb1d3459facfe3a8df2a91ed7 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:02.791481 11254 raft_consensus.cc:2781] T fcf651adb1d3459facfe3a8df2a91ed7 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:02.791795 11254 raft_consensus.cc:482] T fcf651adb1d3459facfe3a8df2a91ed7 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:02.792099 11254 raft_consensus.cc:3037] T fcf651adb1d3459facfe3a8df2a91ed7 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:02.802626 11218 raft_consensus.cc:482] T 5b0345252c3c496db2987c30adce3da7 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:02.802757 10371 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet 85a0ad61b7cb4e11b30e84ddc4655b64 (table default.Table_67493ad72aac4bbca26a6a322a60e5cd [id=12285914811f4220822b47a2a62b01ab]). Expected version 2 got 0
I20260924 00:21:02.803083 11218 raft_consensus.cc:504] T 5b0345252c3c496db2987c30adce3da7 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:02.804886 11218 leader_election.cc:288] T 5b0345252c3c496db2987c30adce3da7 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:21:02.808382 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b0345252c3c496db2987c30adce3da7" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:02.809154 10654 raft_consensus.cc:2454] T 5b0345252c3c496db2987c30adce3da7 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
W20260924 00:21:02.810099 10368 catalog_manager.cc:5687] Failed to send AlterTable RPC for tablet 195b033091b543b58ee08417eae2c0e6 (table default.Table_ca083375f0df4255b8654ed88040dfcd [id=f79ca8d11aa04d5f8f72d58b798ed4ea], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 195b033091b543b58ee08417eae2c0e6 consensus state has no leader
I20260924 00:21:02.810145 10876 leader_election.cc:302] T 5b0345252c3c496db2987c30adce3da7 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:02.810995 11218 raft_consensus.cc:2781] T 5b0345252c3c496db2987c30adce3da7 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:02.811455 11218 raft_consensus.cc:482] T 5b0345252c3c496db2987c30adce3da7 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:02.811815 11218 raft_consensus.cc:3037] T 5b0345252c3c496db2987c30adce3da7 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:02.814924 10634 tablet_service.cc:1514] Processing DeleteTablet for tablet c7e50ac8b7f947b1bfba8cd1cff0ea7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:59 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:02.815762 10630 tablet_service.cc:1514] Processing DeleteTablet for tablet ccb26e0fbc1948bc9a00e9e26c4d1139 with delete_type TABLET_DATA_DELETED (replaced by tablet f9c6a39b2e2842c59b0f996f39d73162) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:02.818778 11306 consensus_queue.cc:1035] T 96960f3061fb43e9939b470f49edf1e9 P b10a9082a37b46f09d581944914c1670 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:02.821429 11041 tablet_bootstrap.cc:492] T 119d83562ef546cab496cd7b44fecc15 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:02.821847 11041 ts_tablet_manager.cc:1397] T 119d83562ef546cab496cd7b44fecc15 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.319s user 0.007s sys 0.003s
I20260924 00:21:02.824074 11041 raft_consensus.cc:348] T 119d83562ef546cab496cd7b44fecc15 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:02.824865 11041 raft_consensus.cc:374] T 119d83562ef546cab496cd7b44fecc15 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:02.825299 11041 raft_consensus.cc:729] T 119d83562ef546cab496cd7b44fecc15 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:02.852653 11041 consensus_queue.cc:260] T 119d83562ef546cab496cd7b44fecc15 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:02.855166 10774 tablet_service.cc:1467] Processing CreateTablet for tablet e937cee22326474598cef5c1d09b59c2 (DEFAULT_TABLE table=default.Table_1ebdc131b4ce4446b021aca0fd94ae99 [id=54d197b284b34dc0a5eea440b0b1e5de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:02.855938 11041 ts_tablet_manager.cc:1428] T 119d83562ef546cab496cd7b44fecc15 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.034s user 0.006s sys 0.000s
I20260924 00:21:02.856106 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e937cee22326474598cef5c1d09b59c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:02.863868 11040 tablet_bootstrap.cc:654] T f294831873d548e4b9201c1bbb964096 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:02.865005 10371 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_0dc486ba6e6d4091912ecb311b5dbd33"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:02.866521 10371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0dc486ba6e6d4091912ecb311b5dbd33 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:02.877899 10370 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_7d1f0174b0c74f7bb29facca7238f8b0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:02.879371 10370 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7d1f0174b0c74f7bb29facca7238f8b0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260924 00:21:02.884711 10592 leader_election.cc:341] T 772fb3d6180e499daf94ceeb9b26048d P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:02.885605 10592 leader_election.cc:302] T 772fb3d6180e499daf94ceeb9b26048d P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:02.840996 11218 raft_consensus.cc:504] T 5b0345252c3c496db2987c30adce3da7 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:02.893536 11306 raft_consensus.cc:2726] T 772fb3d6180e499daf94ceeb9b26048d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:02.911703 11218 leader_election.cc:288] T 5b0345252c3c496db2987c30adce3da7 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:21:02.911778 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b0345252c3c496db2987c30adce3da7" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:21:02.912364 10654 raft_consensus.cc:3037] T 5b0345252c3c496db2987c30adce3da7 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:02.912514 11218 raft_consensus.cc:482] T 59c077eb68794340ac900dbedc7056ee P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:02.912886 11218 raft_consensus.cc:504] T 59c077eb68794340ac900dbedc7056ee P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:02.914707 11051 ts_tablet_manager.cc:1918] T 86374d72238d42c29542e649a7a0200d P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:02.915040 11051 log.cc:1196] T 86374d72238d42c29542e649a7a0200d P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/86374d72238d42c29542e649a7a0200d
I20260924 00:21:02.915628 11051 ts_tablet_manager.cc:1939] T 86374d72238d42c29542e649a7a0200d P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:02.916625 11039 tablet_bootstrap.cc:492] T 666755ec87bb496db75e43ec9204ec5b P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:02.916579 10798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7463fcb18e884b1dbc478ebaa19f6cb4" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:02.917023 11039 ts_tablet_manager.cc:1397] T 666755ec87bb496db75e43ec9204ec5b P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.393s user 0.008s sys 0.007s
I20260924 00:21:02.917713 10370 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_5cf772cb831f4e7e97430617b962db80" } modify_external_catalogs: true
I20260924 00:21:02.918212 10370 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_5cf772cb831f4e7e97430617b962db80" } modify_external_catalogs: true
I20260924 00:21:02.920341 11039 raft_consensus.cc:348] T 666755ec87bb496db75e43ec9204ec5b P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:02.921033 11039 raft_consensus.cc:374] T 666755ec87bb496db75e43ec9204ec5b P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:02.921276 11039 raft_consensus.cc:729] T 666755ec87bb496db75e43ec9204ec5b P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:02.921831 11039 consensus_queue.cc:260] T 666755ec87bb496db75e43ec9204ec5b P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:02.922521 10656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59c077eb68794340ac900dbedc7056ee" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:02.923105 10656 raft_consensus.cc:2454] T 59c077eb68794340ac900dbedc7056ee P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
I20260924 00:21:02.925499 10798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcf651adb1d3459facfe3a8df2a91ed7" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:02.925871 10876 leader_election.cc:302] T 59c077eb68794340ac900dbedc7056ee P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:02.929394 10798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7463fcb18e884b1dbc478ebaa19f6cb4" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:21:02.933931 11218 leader_election.cc:288] T 59c077eb68794340ac900dbedc7056ee P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:21:02.934545 10654 raft_consensus.cc:2454] T 5b0345252c3c496db2987c30adce3da7 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 1.
I20260924 00:21:02.910435 11254 raft_consensus.cc:504] T fcf651adb1d3459facfe3a8df2a91ed7 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:02.937389 11254 leader_election.cc:288] T fcf651adb1d3459facfe3a8df2a91ed7 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:21:02.943372 10876 leader_election.cc:302] T 5b0345252c3c496db2987c30adce3da7 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:02.944458 11256 raft_consensus.cc:2781] T 59c077eb68794340ac900dbedc7056ee P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:02.944885 11041 tablet_bootstrap.cc:492] T ffa011cf8dc446de849bea75a4f20056 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:02.944877 11256 raft_consensus.cc:482] T 59c077eb68794340ac900dbedc7056ee P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:02.945351 11256 raft_consensus.cc:3037] T 59c077eb68794340ac900dbedc7056ee P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:02.952798 10365 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_5333360c3b164ff9a536584172195b56" } new_table_name: "default.Table_0cdc660142f44fb5b77963f9432b4182" modify_external_catalogs: true
I20260924 00:21:02.953531 11049 ts_tablet_manager.cc:1905] T 5d7d52be4f1f4fc29b8172b9fa878f76 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:02.954736 11051 tablet_replica.cc:330] T d44932f8f08e4ea7acc05b8ec511f045 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:02.955422 11051 raft_consensus.cc:2229] T d44932f8f08e4ea7acc05b8ec511f045 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:02.955837 11051 raft_consensus.cc:2258] T d44932f8f08e4ea7acc05b8ec511f045 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:02.956386 11041 tablet_bootstrap.cc:654] T ffa011cf8dc446de849bea75a4f20056 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:02.956161 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b0345252c3c496db2987c30adce3da7" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:02.956350 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 86374d72238d42c29542e649a7a0200d (table default.Table_6a3c73b6e78a4066bb59da236b32ab81 [id=8a99c64662ce4d3bba44aeb578c55c16]) successfully deleted
I20260924 00:21:02.957091 11218 raft_consensus.cc:2781] T 5b0345252c3c496db2987c30adce3da7 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:02.957463 11218 raft_consensus.cc:686] T 5b0345252c3c496db2987c30adce3da7 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Becoming Leader. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Running, Role: LEADER
I20260924 00:21:02.957505 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcf651adb1d3459facfe3a8df2a91ed7" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:21:02.958140 10654 raft_consensus.cc:3037] T fcf651adb1d3459facfe3a8df2a91ed7 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:02.958232 11218 consensus_queue.cc:237] T 5b0345252c3c496db2987c30adce3da7 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:02.958561 10369 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_d3c4523a9e0644f7b0ac485b5b4801db"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:02.960031 10369 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d3c4523a9e0644f7b0ac485b5b4801db in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:02.962347 11039 ts_tablet_manager.cc:1428] T 666755ec87bb496db75e43ec9204ec5b P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.045s user 0.000s sys 0.004s
I20260924 00:21:02.963248 11039 tablet_bootstrap.cc:492] T 195b033091b543b58ee08417eae2c0e6 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:02.966401 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b0345252c3c496db2987c30adce3da7" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:21:02.969835 10372 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_e86eeb97d87d4f9ea7edfe269cc1d790" } new_table_name: "default.Table_9670783a92074b639ecb0d6c374811e5" modify_external_catalogs: true
I20260924 00:21:02.974678 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59c077eb68794340ac900dbedc7056ee" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:02.978107 11051 ts_tablet_manager.cc:1905] T d44932f8f08e4ea7acc05b8ec511f045 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:02.979365 10367 master_service.cc:946] ReplaceTablet: received request to replace tablet 59a0a950a8144e99b2ed47be9a567d9a from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:02.981494 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcf651adb1d3459facfe3a8df2a91ed7" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:21:02.983721 10778 tablet_service.cc:1467] Processing CreateTablet for tablet 675b9e33edcf4d4ea7fa428b476e3945 (DEFAULT_TABLE table=default.Table_ee73693b7a6e46a3af53f7828da00f9e [id=9359b05533064c38b9bd589c0bb06471]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:02.984699 10778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 675b9e33edcf4d4ea7fa428b476e3945. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:02.992691 10373 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_3078c59b2af247ab91ea35e4f8bf9b9c" } modify_external_catalogs: true
I20260924 00:21:02.993309 10373 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_3078c59b2af247ab91ea35e4f8bf9b9c" } modify_external_catalogs: true
I20260924 00:21:02.952533 11256 raft_consensus.cc:504] T 59c077eb68794340ac900dbedc7056ee P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:03.004249 11256 leader_election.cc:288] T 59c077eb68794340ac900dbedc7056ee P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:21:03.004374 10655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59c077eb68794340ac900dbedc7056ee" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:21:03.004963 10655 raft_consensus.cc:3037] T 59c077eb68794340ac900dbedc7056ee P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
W20260924 00:21:03.005661 10365 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cf36d53828eb421d87b52b3e4cb1d748 has no consensus state
I20260924 00:21:03.010905 11039 tablet_bootstrap.cc:654] T 195b033091b543b58ee08417eae2c0e6 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
W20260924 00:21:03.014320 10372 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6bcfd729e9bd4040b3384ab24164e80d consensus state has no leader
I20260924 00:21:03.017988 10365 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_14e49ca823664cc38f51183bc9bdc0ac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:03.020232 10365 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_14e49ca823664cc38f51183bc9bdc0ac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:03.032415 11308 raft_consensus.cc:482] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:03.033000 11308 raft_consensus.cc:504] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:03.049610 10369 master_service.cc:946] ReplaceTablet: received request to replace tablet 8a9443159a334093b4135364057e0c8b from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:03.050959 11040 tablet_bootstrap.cc:492] T f294831873d548e4b9201c1bbb964096 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:03.051370 11040 ts_tablet_manager.cc:1397] T f294831873d548e4b9201c1bbb964096 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.310s user 0.003s sys 0.006s
I20260924 00:21:03.043000 10655 raft_consensus.cc:2454] T 59c077eb68794340ac900dbedc7056ee P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 1.
I20260924 00:21:03.053774 10876 leader_election.cc:302] T 59c077eb68794340ac900dbedc7056ee P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:03.054867 11323 raft_consensus.cc:2781] T 59c077eb68794340ac900dbedc7056ee P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:03.055425 11323 raft_consensus.cc:686] T 59c077eb68794340ac900dbedc7056ee P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Becoming Leader. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Running, Role: LEADER
I20260924 00:21:03.055688 10654 raft_consensus.cc:2454] T fcf651adb1d3459facfe3a8df2a91ed7 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 1.
I20260924 00:21:03.056224 11323 consensus_queue.cc:237] T 59c077eb68794340ac900dbedc7056ee P c8626e300d0644b0908797ad61bd7cef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:03.077247 10773 tablet_service.cc:1514] Processing DeleteTablet for tablet 7463fcb18e884b1dbc478ebaa19f6cb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:02 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:03.079576 10776 tablet_service.cc:1514] Processing DeleteTablet for tablet 86374d72238d42c29542e649a7a0200d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:58 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:03.065935 10940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7e50ac8b7f947b1bfba8cd1cff0ea7e" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:03.102253 10368 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet 96960f3061fb43e9939b470f49edf1e9 (table default.Table_da80d8b26b9b48c79a00dd08f123f4f4 [id=6251651eb5944ee086dd564d86204148]). Expected version 1 got 0
I20260924 00:21:03.102156 11332 tablet.cc:1722] T 1ec36d59b5114feea144bcb72794136c P c8626e300d0644b0908797ad61bd7cef: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260924 00:21:03.103008 10368 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet 1ec36d59b5114feea144bcb72794136c (table default.Table_b1726c09065e4fbaa2e6799b3ebdbc74 [id=4dbb68e0a32349ad8da2945db650584d]). Expected version 1 got 0
I20260924 00:21:03.106197 10371 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 1ec36d59b5114feea144bcb72794136c (table default.Table_b1726c09065e4fbaa2e6799b3ebdbc74 [id=4dbb68e0a32349ad8da2945db650584d]). Expected version 1 got 0
I20260924 00:21:03.106756 10371 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 85a0ad61b7cb4e11b30e84ddc4655b64 (table default.Table_67493ad72aac4bbca26a6a322a60e5cd [id=12285914811f4220822b47a2a62b01ab]). Expected version 2 got 0
I20260924 00:21:03.074016 11039 tablet_bootstrap.cc:492] T 195b033091b543b58ee08417eae2c0e6 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
W20260924 00:21:03.080279 10878 leader_election.cc:341] T 7463fcb18e884b1dbc478ebaa19f6cb4 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
W20260924 00:21:03.108001 10878 leader_election.cc:341] T fcf651adb1d3459facfe3a8df2a91ed7 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
W20260924 00:21:03.108736 10878 leader_election.cc:341] T 7463fcb18e884b1dbc478ebaa19f6cb4 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:03.109604 10374 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 1ec36d59b5114feea144bcb72794136c (table default.Table_b1726c09065e4fbaa2e6799b3ebdbc74 [id=4dbb68e0a32349ad8da2945db650584d]). Expected version 1 got 0
W20260924 00:21:03.109668 10878 leader_election.cc:341] T 5b0345252c3c496db2987c30adce3da7 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:03.092389 11051 ts_tablet_manager.cc:1918] T d44932f8f08e4ea7acc05b8ec511f045 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260924 00:21:03.110431 10878 leader_election.cc:341] T 5b0345252c3c496db2987c30adce3da7 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:03.067633 11308 leader_election.cc:288] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:03.074458 10655 raft_consensus.cc:1261] T f294831873d548e4b9201c1bbb964096 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Refusing update from remote peer c8626e300d0644b0908797ad61bd7cef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:21:03.110865 10876 leader_election.cc:302] T fcf651adb1d3459facfe3a8df2a91ed7 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
W20260924 00:21:03.111171 10878 leader_election.cc:341] T 59c077eb68794340ac900dbedc7056ee P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
W20260924 00:21:03.111931 10878 leader_election.cc:341] T fcf651adb1d3459facfe3a8df2a91ed7 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:03.112040 11218 raft_consensus.cc:2781] T fcf651adb1d3459facfe3a8df2a91ed7 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:03.112720 11324 consensus_queue.cc:1035] T f294831873d548e4b9201c1bbb964096 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Connected to new peer: Peer: permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 00:21:03.113320 10919 tablet_service.cc:1467] Processing CreateTablet for tablet cf36d53828eb421d87b52b3e4cb1d748 (DEFAULT_TABLE table=default.Table_0cdc660142f44fb5b77963f9432b4182 [id=15ae26a9e629499e9c632282502ed104]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.115255 11051 log.cc:1196] T d44932f8f08e4ea7acc05b8ec511f045 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/d44932f8f08e4ea7acc05b8ec511f045
I20260924 00:21:03.114876 11218 raft_consensus.cc:686] T fcf651adb1d3459facfe3a8df2a91ed7 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Becoming Leader. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Running, Role: LEADER
I20260924 00:21:03.115952 11051 ts_tablet_manager.cc:1939] T d44932f8f08e4ea7acc05b8ec511f045 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:03.116485 11218 consensus_queue.cc:237] T fcf651adb1d3459facfe3a8df2a91ed7 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:03.124198 10920 tablet_service.cc:1467] Processing CreateTablet for tablet 3c850d0474414d2486cd7bbd6568e936 (DEFAULT_TABLE table=default.Table_294eebbf563b4b24a4f75064b4e9233b [id=c98b19807d814947b152c5faba758be1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.127158 10365 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_aed2460280354a9b94c52d37815f3347"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:03.129895 10365 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aed2460280354a9b94c52d37815f3347 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:03.131541 11039 ts_tablet_manager.cc:1397] T 195b033091b543b58ee08417eae2c0e6 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.169s user 0.006s sys 0.006s
I20260924 00:21:03.132696 10374 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_dc51cf031b5e41ccb485d5db42ec823a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:03.072894 11329 tablet.cc:1722] T 1ec36d59b5114feea144bcb72794136c P 48fbd603abe146c585d15196e1b5be08: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260924 00:21:03.134459 10374 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dc51cf031b5e41ccb485d5db42ec823a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:03.099421 11330 tablet.cc:1722] T 1ec36d59b5114feea144bcb72794136c P b10a9082a37b46f09d581944914c1670: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260924 00:21:03.100347 10589 leader_election.cc:341] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:03.137672 11041 tablet_bootstrap.cc:492] T ffa011cf8dc446de849bea75a4f20056 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:03.138172 11041 ts_tablet_manager.cc:1397] T ffa011cf8dc446de849bea75a4f20056 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.193s user 0.002s sys 0.011s
I20260924 00:21:03.139094 10629 tablet_service.cc:1467] Processing CreateTablet for tablet cf36d53828eb421d87b52b3e4cb1d748 (DEFAULT_TABLE table=default.Table_0cdc660142f44fb5b77963f9432b4182 [id=15ae26a9e629499e9c632282502ed104]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.140551 10629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf36d53828eb421d87b52b3e4cb1d748. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.119030 11040 raft_consensus.cc:348] T f294831873d548e4b9201c1bbb964096 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:03.142643 11040 raft_consensus.cc:374] T f294831873d548e4b9201c1bbb964096 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:03.142910 11040 raft_consensus.cc:729] T f294831873d548e4b9201c1bbb964096 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:03.143692 11051 tablet_replica.cc:330] T 7463fcb18e884b1dbc478ebaa19f6cb4 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:03.144541 11051 raft_consensus.cc:2229] T 7463fcb18e884b1dbc478ebaa19f6cb4 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:21:03.145219 11051 pending_rounds.cc:62] T 7463fcb18e884b1dbc478ebaa19f6cb4 P c8626e300d0644b0908797ad61bd7cef: Trying to abort 1 pending ops.
I20260924 00:21:03.145294 10636 tablet_service.cc:1467] Processing CreateTablet for tablet 3c850d0474414d2486cd7bbd6568e936 (DEFAULT_TABLE table=default.Table_294eebbf563b4b24a4f75064b4e9233b [id=c98b19807d814947b152c5faba758be1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.145469 11051 pending_rounds.cc:68] T 7463fcb18e884b1dbc478ebaa19f6cb4 P c8626e300d0644b0908797ad61bd7cef: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332697140375941120 op_type: NO_OP noop_request { }
I20260924 00:21:03.145807 11051 raft_consensus.cc:2866] T 7463fcb18e884b1dbc478ebaa19f6cb4 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260924 00:21:03.146090 11051 raft_consensus.cc:2258] T 7463fcb18e884b1dbc478ebaa19f6cb4 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:03.146234 10636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c850d0474414d2486cd7bbd6568e936. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.147953 11051 ts_tablet_manager.cc:1905] T 7463fcb18e884b1dbc478ebaa19f6cb4 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:03.148357 11039 raft_consensus.cc:348] T 195b033091b543b58ee08417eae2c0e6 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:03.149010 11039 raft_consensus.cc:374] T 195b033091b543b58ee08417eae2c0e6 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:03.149238 11039 raft_consensus.cc:729] T 195b033091b543b58ee08417eae2c0e6 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:03.149911 10919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf36d53828eb421d87b52b3e4cb1d748. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.157016 10367 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 59a0a950a8144e99b2ed47be9a567d9a
I20260924 00:21:03.157131 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet d44932f8f08e4ea7acc05b8ec511f045 (table default.Table_321e6959ada84ccea6413114309bc3b8 [id=8092ca5df917480aba7e26a4cedf9fbd]) successfully deleted
I20260924 00:21:03.158334 10367 catalog_manager.cc:6565] ReplaceTablet: tablet 59a0a950a8144e99b2ed47be9a567d9a deleted and replaced by tablet d4e120acff3e4c30aed813b0b24fbfee
I20260924 00:21:03.160631 10372 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 96960f3061fb43e9939b470f49edf1e9 (table default.Table_da80d8b26b9b48c79a00dd08f123f4f4 [id=6251651eb5944ee086dd564d86204148]). Expected version 1 got 0
I20260924 00:21:03.163107 10372 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 85a0ad61b7cb4e11b30e84ddc4655b64 (table default.Table_67493ad72aac4bbca26a6a322a60e5cd [id=12285914811f4220822b47a2a62b01ab]). Expected version 2 got 0
I20260924 00:21:03.163723 10372 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 119d83562ef546cab496cd7b44fecc15 (table default.Table_76ffb857cf034b6f80598edd90dad953 [id=5fb5e1e90c9440f385e7b04520bcba9c]). Expected version 1 got 0
W20260924 00:21:03.171514 10361 consensus_peers.cc:487] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 -> Peer f8bd0b3184f84f5f8da10c9c21472f92 (127.10.4.124:35345): Couldn't send request to peer f8bd0b3184f84f5f8da10c9c21472f92. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.124:35345: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260924 00:21:03.172456 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7e50ac8b7f947b1bfba8cd1cff0ea7e" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:03.140775 11041 raft_consensus.cc:348] T ffa011cf8dc446de849bea75a4f20056 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:03.174147 11041 raft_consensus.cc:374] T ffa011cf8dc446de849bea75a4f20056 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:03.174048 10635 tablet_service.cc:1514] Processing DeleteTablet for tablet 59a0a950a8144e99b2ed47be9a567d9a with delete_type TABLET_DATA_DELETED (replaced by tablet d4e120acff3e4c30aed813b0b24fbfee) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:03.174513 11041 raft_consensus.cc:729] T ffa011cf8dc446de849bea75a4f20056 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:03.175101 11041 consensus_queue.cc:260] T ffa011cf8dc446de849bea75a4f20056 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:03.181695 11041 ts_tablet_manager.cc:1428] T ffa011cf8dc446de849bea75a4f20056 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.043s user 0.006s sys 0.000s
I20260924 00:21:03.182925 11041 tablet_bootstrap.cc:492] T b06bf1e4691441769e162aa5404d4656 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:03.190326 10635 tablet_service.cc:1467] Processing CreateTablet for tablet a62b85ebb2b84ae19cad3e3e2dd07a59 (DEFAULT_TABLE table=default.Table_ab9660d8c1a5491c8fee2f43483105a7 [id=3f6c757389bb4213b584f3e5a9ac7ede]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.190613 11335 tablet_replica.cc:330] T 86374d72238d42c29542e649a7a0200d P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:03.191326 10635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a62b85ebb2b84ae19cad3e3e2dd07a59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.191412 11335 raft_consensus.cc:2229] T 86374d72238d42c29542e649a7a0200d P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:21:03.161741 11039 consensus_queue.cc:260] T 195b033091b543b58ee08417eae2c0e6 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:03.178375 10922 tablet_service.cc:1514] Processing DeleteTablet for tablet 59a0a950a8144e99b2ed47be9a567d9a with delete_type TABLET_DATA_DELETED (replaced by tablet d4e120acff3e4c30aed813b0b24fbfee) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:03.192221 11335 pending_rounds.cc:62] T 86374d72238d42c29542e649a7a0200d P 48fbd603abe146c585d15196e1b5be08: Trying to abort 1 pending ops.
I20260924 00:21:03.192432 11335 pending_rounds.cc:68] T 86374d72238d42c29542e649a7a0200d P 48fbd603abe146c585d15196e1b5be08: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332697139421372416 op_type: NO_OP noop_request { }
I20260924 00:21:03.192806 11335 raft_consensus.cc:2866] T 86374d72238d42c29542e649a7a0200d P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260924 00:21:03.193177 11335 raft_consensus.cc:2258] T 86374d72238d42c29542e649a7a0200d P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:03.201297 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59c077eb68794340ac900dbedc7056ee" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:21:03.205587 10773 tablet_service.cc:1514] Processing DeleteTablet for tablet 59a0a950a8144e99b2ed47be9a567d9a with delete_type TABLET_DATA_DELETED (replaced by tablet d4e120acff3e4c30aed813b0b24fbfee) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:03.205549 10776 tablet_service.cc:1467] Processing CreateTablet for tablet cf36d53828eb421d87b52b3e4cb1d748 (DEFAULT_TABLE table=default.Table_0cdc660142f44fb5b77963f9432b4182 [id=15ae26a9e629499e9c632282502ed104]), partition=RANGE (key) PARTITION UNBOUNDED
W20260924 00:21:03.206020 10592 leader_election.cc:341] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:03.206408 10592 leader_election.cc:302] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:03.206622 10776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf36d53828eb421d87b52b3e4cb1d748. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.206815 10773 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d7d52be4f1f4fc29b8172b9fa878f76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:57 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:03.206974 11308 raft_consensus.cc:2726] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:03.207885 10774 tablet_service.cc:1467] Processing CreateTablet for tablet 3c850d0474414d2486cd7bbd6568e936 (DEFAULT_TABLE table=default.Table_294eebbf563b4b24a4f75064b4e9233b [id=c98b19807d814947b152c5faba758be1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.208837 10630 tablet_service.cc:1467] Processing CreateTablet for tablet 5b43ce1fed0749b8b076b9b026914e07 (DEFAULT_TABLE table=default.Table_9608bad42525495eaa4df537b37969e7 [id=aa74d52589984aec8e6d296e5dffd623]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.209873 10630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b43ce1fed0749b8b076b9b026914e07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.210881 11335 ts_tablet_manager.cc:1905] T 86374d72238d42c29542e649a7a0200d P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:03.208853 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c850d0474414d2486cd7bbd6568e936. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.223356 10773 tablet_service.cc:1467] Processing CreateTablet for tablet 5b43ce1fed0749b8b076b9b026914e07 (DEFAULT_TABLE table=default.Table_9608bad42525495eaa4df537b37969e7 [id=aa74d52589984aec8e6d296e5dffd623]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.224486 10773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b43ce1fed0749b8b076b9b026914e07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.226600 10367 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_da80d8b26b9b48c79a00dd08f123f4f4" } new_table_name: "default.Table_477c0f1cce9a4c089551e4e767ba7ae0" modify_external_catalogs: true
I20260924 00:21:03.229620 10778 tablet_service.cc:1467] Processing CreateTablet for tablet a62b85ebb2b84ae19cad3e3e2dd07a59 (DEFAULT_TABLE table=default.Table_ab9660d8c1a5491c8fee2f43483105a7 [id=3f6c757389bb4213b584f3e5a9ac7ede]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.230690 10778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a62b85ebb2b84ae19cad3e3e2dd07a59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.172492 11040 consensus_queue.cc:260] T f294831873d548e4b9201c1bbb964096 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:03.234479 10371 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_a7b8969040dd4ef5ad547c617f03d471" } new_table_name: "default.Table_382099cf77a24bb8bd6fe084926431c9" modify_external_catalogs: true
I20260924 00:21:03.235832 10366 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_6496e9c31f6349e78e6f469660e561d6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:03.237867 10366 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6496e9c31f6349e78e6f469660e561d6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:03.238591 11040 ts_tablet_manager.cc:1428] T f294831873d548e4b9201c1bbb964096 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.187s user 0.007s sys 0.000s
I20260924 00:21:03.239408 11040 tablet_bootstrap.cc:492] T 5b0345252c3c496db2987c30adce3da7 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:03.247473 10368 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet f294831873d548e4b9201c1bbb964096 (table default.Table_de36c9f702024914965bc6b371eda68e [id=e2cdad6dfe7846ebb3c14f5611cf0f18]). Expected version 1 got 0
I20260924 00:21:03.239789 11049 ts_tablet_manager.cc:1918] T 5d7d52be4f1f4fc29b8172b9fa878f76 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260924 00:21:03.250619 10878 leader_election.cc:341] T 59c077eb68794340ac900dbedc7056ee P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:03.251067 10920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c850d0474414d2486cd7bbd6568e936. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.255162 10372 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet 85a0ad61b7cb4e11b30e84ddc4655b64 (table default.Table_67493ad72aac4bbca26a6a322a60e5cd [id=12285914811f4220822b47a2a62b01ab]). Expected version 2 got 0
I20260924 00:21:03.258293 10370 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 59c077eb68794340ac900dbedc7056ee
I20260924 00:21:03.261035 10369 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 8a9443159a334093b4135364057e0c8b
I20260924 00:21:03.261073 10917 tablet_service.cc:1467] Processing CreateTablet for tablet a62b85ebb2b84ae19cad3e3e2dd07a59 (DEFAULT_TABLE table=default.Table_ab9660d8c1a5491c8fee2f43483105a7 [id=3f6c757389bb4213b584f3e5a9ac7ede]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.262421 10369 catalog_manager.cc:6565] ReplaceTablet: tablet 8a9443159a334093b4135364057e0c8b deleted and replaced by tablet a53f06276dbb4d9f97a154a8ceb0278d
I20260924 00:21:03.263213 10777 tablet_service.cc:1514] Processing DeleteTablet for tablet 59c077eb68794340ac900dbedc7056ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:02 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:03.265615 10777 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a9443159a334093b4135364057e0c8b with delete_type TABLET_DATA_DELETED (replaced by tablet a53f06276dbb4d9f97a154a8ceb0278d) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:03.267421 11308 raft_consensus.cc:482] T 119d83562ef546cab496cd7b44fecc15 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:03.267851 11308 raft_consensus.cc:504] T 119d83562ef546cab496cd7b44fecc15 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:03.269598 11308 leader_election.cc:288] T 119d83562ef546cab496cd7b44fecc15 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:03.270741 10630 tablet_service.cc:1514] Processing DeleteTablet for tablet 59c077eb68794340ac900dbedc7056ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:02 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:03.271813 10630 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a9443159a334093b4135364057e0c8b with delete_type TABLET_DATA_DELETED (replaced by tablet a53f06276dbb4d9f97a154a8ceb0278d) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:03.273427 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "119d83562ef546cab496cd7b44fecc15" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:03.276682 10592 leader_election.cc:341] T 119d83562ef546cab496cd7b44fecc15 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:03.280526 10777 tablet_service.cc:1467] Processing CreateTablet for tablet 717759eb61674c89b05b8d2b0dc34979 (DEFAULT_TABLE table=default.Table_9b991221b38645378eda8c1403ac0ce7 [id=b22dcf8c1c4346bebde8d0b9af8cf77c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.282126 10777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 717759eb61674c89b05b8d2b0dc34979. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.292975 10779 tablet_service.cc:1467] Processing CreateTablet for tablet 5107b763acf04df88553ba0af2757570 (DEFAULT_TABLE table=default.Table_ff0bbc1b70c2417e9c69a7f7492b2981 [id=667f4349af3f4eb28461e33f65beeb1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.294073 10779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5107b763acf04df88553ba0af2757570. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.310004 10917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a62b85ebb2b84ae19cad3e3e2dd07a59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.320473 11041 tablet_bootstrap.cc:654] T b06bf1e4691441769e162aa5404d4656 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:03.322655 11039 ts_tablet_manager.cc:1428] T 195b033091b543b58ee08417eae2c0e6 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.191s user 0.007s sys 0.005s
I20260924 00:21:03.323601 11039 tablet_bootstrap.cc:492] T f77aea0aad5a430f941aaec0a1c19640 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:03.329015 10630 tablet_service.cc:1467] Processing CreateTablet for tablet 5107b763acf04df88553ba0af2757570 (DEFAULT_TABLE table=default.Table_ff0bbc1b70c2417e9c69a7f7492b2981 [id=667f4349af3f4eb28461e33f65beeb1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.330084 10630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5107b763acf04df88553ba0af2757570. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.330016 10775 tablet_service.cc:1467] Processing CreateTablet for tablet 9319f9caadb04067a2e9700b5019ed37 (DEFAULT_TABLE table=default.Table_4c54324167a8434a90e8dcb1354111f9 [id=9da4000ba42645d69f7f3c5fc3a941c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.332700 10775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9319f9caadb04067a2e9700b5019ed37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.337813 10370 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_f19e1bc3467b4ac9943c4873aba523de"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:03.339243 10370 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f19e1bc3467b4ac9943c4873aba523de in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:03.344283 10780 tablet_service.cc:1467] Processing CreateTablet for tablet 98926dd3850d40938b67779d5fd9a1db (DEFAULT_TABLE table=default.Table_22740cf31fbe40a58c31252f63417ad3 [id=74b4a09bdb464f27ae5d19a6a53ece5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.345291 10780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98926dd3850d40938b67779d5fd9a1db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.353812 11040 tablet_bootstrap.cc:654] T 5b0345252c3c496db2987c30adce3da7 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:03.332062 11049 log.cc:1196] T 5d7d52be4f1f4fc29b8172b9fa878f76 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/5d7d52be4f1f4fc29b8172b9fa878f76
I20260924 00:21:03.345216 10635 tablet_service.cc:1467] Processing CreateTablet for tablet 717759eb61674c89b05b8d2b0dc34979 (DEFAULT_TABLE table=default.Table_9b991221b38645378eda8c1403ac0ce7 [id=b22dcf8c1c4346bebde8d0b9af8cf77c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.357311 10635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 717759eb61674c89b05b8d2b0dc34979. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.357949 11049 ts_tablet_manager.cc:1939] T 5d7d52be4f1f4fc29b8172b9fa878f76 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:03.361178 11041 tablet_bootstrap.cc:492] T b06bf1e4691441769e162aa5404d4656 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:03.361634 11041 ts_tablet_manager.cc:1397] T b06bf1e4691441769e162aa5404d4656 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.179s user 0.011s sys 0.002s
I20260924 00:21:03.363762 11041 raft_consensus.cc:348] T b06bf1e4691441769e162aa5404d4656 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:03.364290 11049 tablet_replica.cc:330] T 86374d72238d42c29542e649a7a0200d P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:03.364398 11041 raft_consensus.cc:374] T b06bf1e4691441769e162aa5404d4656 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:03.364634 11041 raft_consensus.cc:729] T b06bf1e4691441769e162aa5404d4656 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:03.365027 11049 raft_consensus.cc:2229] T 86374d72238d42c29542e649a7a0200d P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:03.381469 10774 tablet_service.cc:1467] Processing CreateTablet for tablet 4c2a729a802e4abe9ab4f989a40198be (DEFAULT_TABLE table=default.Table_dca718fc1c1a4e518bbc52483fdd7a09 [id=136a32ddaf2a4d2fbdddecf3ea95970c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.382506 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c2a729a802e4abe9ab4f989a40198be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.365366 11041 consensus_queue.cc:260] T b06bf1e4691441769e162aa5404d4656 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:03.386734 10772 tablet_service.cc:1467] Processing CreateTablet for tablet cd30f52583124543b5d616ef94b31831 (DEFAULT_TABLE table=default.Table_3078c59b2af247ab91ea35e4f8bf9b9c [id=89a81661022b40eda6eecd81008a3107]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.394099 10772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd30f52583124543b5d616ef94b31831. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.401005 11335 ts_tablet_manager.cc:1918] T 86374d72238d42c29542e649a7a0200d P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:03.401472 11335 log.cc:1196] T 86374d72238d42c29542e649a7a0200d P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/86374d72238d42c29542e649a7a0200d
I20260924 00:21:03.402256 11335 ts_tablet_manager.cc:1939] T 86374d72238d42c29542e649a7a0200d P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:03.405407 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 86374d72238d42c29542e649a7a0200d (table default.Table_6a3c73b6e78a4066bb59da236b32ab81 [id=8a99c64662ce4d3bba44aeb578c55c16]) successfully deleted
W20260924 00:21:03.414458 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 7463fcb18e884b1dbc478ebaa19f6cb4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7463fcb18e884b1dbc478ebaa19f6cb4 already in progress: creating tablet
W20260924 00:21:03.416256 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 59a0a950a8144e99b2ed47be9a567d9a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 59a0a950a8144e99b2ed47be9a567d9a already in progress: creating tablet
I20260924 00:21:03.417379 11335 tablet_replica.cc:330] T 5d7d52be4f1f4fc29b8172b9fa878f76 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:03.418205 10773 tablet_service.cc:1467] Processing CreateTablet for tablet 57ae3d10846a4aaa84eba81b13aafd44 (DEFAULT_TABLE table=default.Table_29e523da31634984b775f31084e82c99 [id=6d2a4bce1272466c8a00217643c0c9f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.419305 10773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57ae3d10846a4aaa84eba81b13aafd44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.418190 11335 raft_consensus.cc:2229] T 5d7d52be4f1f4fc29b8172b9fa878f76 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:03.425642 11335 raft_consensus.cc:2258] T 5d7d52be4f1f4fc29b8172b9fa878f76 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:03.428647 11335 ts_tablet_manager.cc:1905] T 5d7d52be4f1f4fc29b8172b9fa878f76 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:03.311362 11256 raft_consensus.cc:482] T 666755ec87bb496db75e43ec9204ec5b P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:03.449729 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 5d7d52be4f1f4fc29b8172b9fa878f76 (table default.Table_451536c1b7444188b1dcab7ce70d663f [id=fde3781ccd3748f2acc7ac5f3f5064bf]) successfully deleted
W20260924 00:21:03.450736 10359 catalog_manager.cc:4808] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet ae0c9b785195491987dd8fe397d704e0 because the tablet was not found. No further retry: Not found: Tablet not found: ae0c9b785195491987dd8fe397d704e0
W20260924 00:21:03.451326 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet f3432ac96da94d5099bcf9764c2b32e1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f3432ac96da94d5099bcf9764c2b32e1 already in progress: creating tablet
I20260924 00:21:03.469532 10368 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_9b991221b38645378eda8c1403ac0ce7" } modify_external_catalogs: true
I20260924 00:21:03.470095 10368 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_9b991221b38645378eda8c1403ac0ce7" } modify_external_catalogs: true
W20260924 00:21:03.470563 10878 consensus_peers.cc:487] T f294831873d548e4b9201c1bbb964096 P c8626e300d0644b0908797ad61bd7cef -> Peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Couldn't send request to peer 48fbd603abe146c585d15196e1b5be08. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260924 00:21:03.470920 10878 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:35359, user_credentials={real_user=slave}} blocked reactor thread for 210279us
I20260924 00:21:03.473686 11335 ts_tablet_manager.cc:1918] T 5d7d52be4f1f4fc29b8172b9fa878f76 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:03.474042 11335 log.cc:1196] T 5d7d52be4f1f4fc29b8172b9fa878f76 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/5d7d52be4f1f4fc29b8172b9fa878f76
I20260924 00:21:03.474694 11335 ts_tablet_manager.cc:1939] T 5d7d52be4f1f4fc29b8172b9fa878f76 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:03.477142 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 5d7d52be4f1f4fc29b8172b9fa878f76 (table default.Table_451536c1b7444188b1dcab7ce70d663f [id=fde3781ccd3748f2acc7ac5f3f5064bf]) successfully deleted
W20260924 00:21:03.485754 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 59c077eb68794340ac900dbedc7056ee because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 59c077eb68794340ac900dbedc7056ee already in progress: creating tablet
I20260924 00:21:03.488107 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "119d83562ef546cab496cd7b44fecc15" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
W20260924 00:21:03.488400 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 8a9443159a334093b4135364057e0c8b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8a9443159a334093b4135364057e0c8b already in progress: creating tablet
I20260924 00:21:03.489832 10922 tablet_service.cc:1467] Processing CreateTablet for tablet 5b43ce1fed0749b8b076b9b026914e07 (DEFAULT_TABLE table=default.Table_9608bad42525495eaa4df537b37969e7 [id=aa74d52589984aec8e6d296e5dffd623]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.491052 10922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b43ce1fed0749b8b076b9b026914e07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.468079 11256 raft_consensus.cc:504] T 666755ec87bb496db75e43ec9204ec5b P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:03.514511 11256 leader_election.cc:288] T 666755ec87bb496db75e43ec9204ec5b P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
W20260924 00:21:03.535444 10876 leader_election.cc:341] T 666755ec87bb496db75e43ec9204ec5b P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: 666755ec87bb496db75e43ec9204ec5b
I20260924 00:21:03.566145 11254 raft_consensus.cc:482] T 195b033091b543b58ee08417eae2c0e6 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:03.566635 11254 raft_consensus.cc:504] T 195b033091b543b58ee08417eae2c0e6 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:03.579985 11254 leader_election.cc:288] T 195b033091b543b58ee08417eae2c0e6 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:21:03.585660 11332 tablet.cc:1722] T f294831873d548e4b9201c1bbb964096 P c8626e300d0644b0908797ad61bd7cef: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260924 00:21:03.600422 10635 tablet_service.cc:1467] Processing CreateTablet for tablet 9319f9caadb04067a2e9700b5019ed37 (DEFAULT_TABLE table=default.Table_4c54324167a8434a90e8dcb1354111f9 [id=9da4000ba42645d69f7f3c5fc3a941c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.601444 10635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9319f9caadb04067a2e9700b5019ed37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.601358 10636 tablet_service.cc:1467] Processing CreateTablet for tablet cd30f52583124543b5d616ef94b31831 (DEFAULT_TABLE table=default.Table_3078c59b2af247ab91ea35e4f8bf9b9c [id=89a81661022b40eda6eecd81008a3107]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.602362 10636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd30f52583124543b5d616ef94b31831. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.605700 10630 tablet_service.cc:1467] Processing CreateTablet for tablet 98926dd3850d40938b67779d5fd9a1db (DEFAULT_TABLE table=default.Table_22740cf31fbe40a58c31252f63417ad3 [id=74b4a09bdb464f27ae5d19a6a53ece5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.606731 10630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98926dd3850d40938b67779d5fd9a1db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.609681 10629 tablet_service.cc:1467] Processing CreateTablet for tablet 4c2a729a802e4abe9ab4f989a40198be (DEFAULT_TABLE table=default.Table_dca718fc1c1a4e518bbc52483fdd7a09 [id=136a32ddaf2a4d2fbdddecf3ea95970c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.610074 10634 tablet_service.cc:1467] Processing CreateTablet for tablet 57ae3d10846a4aaa84eba81b13aafd44 (DEFAULT_TABLE table=default.Table_29e523da31634984b775f31084e82c99 [id=6d2a4bce1272466c8a00217643c0c9f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.611253 10629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c2a729a802e4abe9ab4f989a40198be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.611773 10634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57ae3d10846a4aaa84eba81b13aafd44. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:21:03.627525 10359 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 6bcfd729e9bd4040b3384ab24164e80d (table default.Table_9670783a92074b639ecb0d6c374811e5 [id=df2a5a74db65431c9ecb750184f64709], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 6bcfd729e9bd4040b3384ab24164e80d consensus state has no leader
I20260924 00:21:03.650846 11039 tablet_bootstrap.cc:654] T f77aea0aad5a430f941aaec0a1c19640 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:03.653406 11049 raft_consensus.cc:2258] T 86374d72238d42c29542e649a7a0200d P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:03.654484 11041 ts_tablet_manager.cc:1428] T b06bf1e4691441769e162aa5404d4656 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.293s user 0.005s sys 0.000s
I20260924 00:21:03.655258 11049 ts_tablet_manager.cc:1905] T 86374d72238d42c29542e649a7a0200d P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260924 00:21:03.660149 10589 leader_election.cc:341] T 119d83562ef546cab496cd7b44fecc15 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:03.660611 10589 leader_election.cc:302] T 119d83562ef546cab496cd7b44fecc15 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:03.665175 11308 raft_consensus.cc:2726] T 119d83562ef546cab496cd7b44fecc15 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:03.655278 11041 tablet_bootstrap.cc:492] T 0d555f059f84478eb805f4ca4534dab4 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:03.668435 11051 ts_tablet_manager.cc:1918] T 7463fcb18e884b1dbc478ebaa19f6cb4 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:03.668794 11051 log.cc:1196] T 7463fcb18e884b1dbc478ebaa19f6cb4 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/7463fcb18e884b1dbc478ebaa19f6cb4
I20260924 00:21:03.669966 11051 ts_tablet_manager.cc:1939] T 7463fcb18e884b1dbc478ebaa19f6cb4 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:03.674841 11041 tablet_bootstrap.cc:654] T 0d555f059f84478eb805f4ca4534dab4 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:03.696996 11040 tablet_bootstrap.cc:492] T 5b0345252c3c496db2987c30adce3da7 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:03.697383 11040 ts_tablet_manager.cc:1397] T 5b0345252c3c496db2987c30adce3da7 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.458s user 0.081s sys 0.126s
I20260924 00:21:03.698601 10374 master_service.cc:946] ReplaceTablet: received request to replace tablet a0f71049f6b3478da3b07ebc9293ce64 from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:03.699589 10373 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet cd30f52583124543b5d616ef94b31831
I20260924 00:21:03.700403 11040 raft_consensus.cc:348] T 5b0345252c3c496db2987c30adce3da7 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:03.701032 11040 raft_consensus.cc:374] T 5b0345252c3c496db2987c30adce3da7 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:03.701256 11040 raft_consensus.cc:729] T 5b0345252c3c496db2987c30adce3da7 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:03.701773 11040 consensus_queue.cc:260] T 5b0345252c3c496db2987c30adce3da7 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:03.725679 11040 ts_tablet_manager.cc:1428] T 5b0345252c3c496db2987c30adce3da7 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.028s user 0.005s sys 0.000s
I20260924 00:21:03.726550 11040 tablet_bootstrap.cc:492] T 59c077eb68794340ac900dbedc7056ee P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:03.783730 10636 tablet_service.cc:1514] Processing DeleteTablet for tablet cd30f52583124543b5d616ef94b31831 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:02 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:03.786273 11040 tablet_bootstrap.cc:654] T 59c077eb68794340ac900dbedc7056ee P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:03.793383 10775 tablet_service.cc:1514] Processing DeleteTablet for tablet cd30f52583124543b5d616ef94b31831 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:02 UTC) from {username='slave'} at 127.0.0.1:35996
W20260924 00:21:03.795218 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet cd30f52583124543b5d616ef94b31831 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cd30f52583124543b5d616ef94b31831 already in progress: creating tablet
I20260924 00:21:03.801618 11041 tablet_bootstrap.cc:492] T 0d555f059f84478eb805f4ca4534dab4 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:03.802096 11041 ts_tablet_manager.cc:1397] T 0d555f059f84478eb805f4ca4534dab4 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.147s user 0.006s sys 0.003s
I20260924 00:21:03.804324 11041 raft_consensus.cc:348] T 0d555f059f84478eb805f4ca4534dab4 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:03.805161 11041 raft_consensus.cc:374] T 0d555f059f84478eb805f4ca4534dab4 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:03.805461 11041 raft_consensus.cc:729] T 0d555f059f84478eb805f4ca4534dab4 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:03.806140 11041 consensus_queue.cc:260] T 0d555f059f84478eb805f4ca4534dab4 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:03.808492 11041 ts_tablet_manager.cc:1428] T 0d555f059f84478eb805f4ca4534dab4 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260924 00:21:03.821369 10366 master_service.cc:946] ReplaceTablet: received request to replace tablet 675b9e33edcf4d4ea7fa428b476e3945 from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:03.822809 11041 tablet_bootstrap.cc:492] T 0980afc2af174da2a8ca2dd414e9825c P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:03.828511 11322 raft_consensus.cc:482] T 0d555f059f84478eb805f4ca4534dab4 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:03.828969 11322 raft_consensus.cc:504] T 0d555f059f84478eb805f4ca4534dab4 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
W20260924 00:21:03.830168 10371 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bf144ab1a6464a4ab02ac0642c5255e3 consensus state has no leader
I20260924 00:21:03.830859 11322 leader_election.cc:288] T 0d555f059f84478eb805f4ca4534dab4 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:03.831625 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d555f059f84478eb805f4ca4534dab4" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:03.831981 10372 catalog_manager.cc:6040] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: default.Table_b1726c09065e4fbaa2e6799b3ebdbc74 [id=4dbb68e0a32349ad8da2945db650584d] alter complete (version 1)
W20260924 00:21:03.834173 10592 leader_election.cc:341] T 0d555f059f84478eb805f4ca4534dab4 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:03.846396 11322 raft_consensus.cc:482] T b06bf1e4691441769e162aa5404d4656 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:03.846938 11322 raft_consensus.cc:504] T b06bf1e4691441769e162aa5404d4656 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:03.847504 11049 ts_tablet_manager.cc:1918] T 86374d72238d42c29542e649a7a0200d P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:03.849687 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b06bf1e4691441769e162aa5404d4656" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:03.851019 10592 leader_election.cc:341] T b06bf1e4691441769e162aa5404d4656 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:03.852182 11322 leader_election.cc:288] T b06bf1e4691441769e162aa5404d4656 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:03.853215 10629 tablet_service.cc:1514] Processing DeleteTablet for tablet ffa011cf8dc446de849bea75a4f20056 with delete_type TABLET_DATA_DELETED (replaced by tablet 2499754460e04c27ac53d52a4a67f557) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:03.848102 11049 log.cc:1196] T 86374d72238d42c29542e649a7a0200d P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/86374d72238d42c29542e649a7a0200d
I20260924 00:21:03.854102 11049 ts_tablet_manager.cc:1939] T 86374d72238d42c29542e649a7a0200d P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:03.854923 11041 tablet_bootstrap.cc:654] T 0980afc2af174da2a8ca2dd414e9825c P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:03.855954 11049 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 128) took 1416 ms. Trace:
I20260924 00:21:03.856244 11049 rpcz_store.cc:276] 0924 00:21:02.439538 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:02.439668 (+ 130us) service_pool.cc:227] Handling call
0924 00:21:03.364155 (+924487us) ts_tablet_manager.cc:1145] Deleting tablet 86374d72238d42c29542e649a7a0200d
0924 00:21:03.364185 (+ 30us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:03.735494 (+371309us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:03.847294 (+111800us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:03.855899 (+ 8605us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":924021}
I20260924 00:21:03.870337 10911 tablet_service.cc:1467] Processing CreateTablet for tablet cd30f52583124543b5d616ef94b31831 (DEFAULT_TABLE table=default.Table_3078c59b2af247ab91ea35e4f8bf9b9c [id=89a81661022b40eda6eecd81008a3107]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.871395 10915 tablet_service.cc:1467] Processing CreateTablet for tablet 717759eb61674c89b05b8d2b0dc34979 (DEFAULT_TABLE table=default.Table_9b991221b38645378eda8c1403ac0ce7 [id=b22dcf8c1c4346bebde8d0b9af8cf77c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.873175 10916 tablet_service.cc:1467] Processing CreateTablet for tablet 5107b763acf04df88553ba0af2757570 (DEFAULT_TABLE table=default.Table_ff0bbc1b70c2417e9c69a7f7492b2981 [id=667f4349af3f4eb28461e33f65beeb1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.877503 10918 tablet_service.cc:1514] Processing DeleteTablet for tablet 59c077eb68794340ac900dbedc7056ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:02 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:03.878290 10911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd30f52583124543b5d616ef94b31831. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.879269 10912 tablet_service.cc:1467] Processing CreateTablet for tablet 4c2a729a802e4abe9ab4f989a40198be (DEFAULT_TABLE table=default.Table_dca718fc1c1a4e518bbc52483fdd7a09 [id=136a32ddaf2a4d2fbdddecf3ea95970c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.879173 10941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d555f059f84478eb805f4ca4534dab4" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:03.882229 10921 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a9443159a334093b4135364057e0c8b with delete_type TABLET_DATA_DELETED (replaced by tablet a53f06276dbb4d9f97a154a8ceb0278d) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:03.884224 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "195b033091b543b58ee08417eae2c0e6" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:03.884836 10654 raft_consensus.cc:2454] T 195b033091b543b58ee08417eae2c0e6 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
I20260924 00:21:03.877548 10939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b06bf1e4691441769e162aa5404d4656" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:03.873260 10913 tablet_service.cc:1467] Processing CreateTablet for tablet 98926dd3850d40938b67779d5fd9a1db (DEFAULT_TABLE table=default.Table_22740cf31fbe40a58c31252f63417ad3 [id=74b4a09bdb464f27ae5d19a6a53ece5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.870333 10914 tablet_service.cc:1467] Processing CreateTablet for tablet 9319f9caadb04067a2e9700b5019ed37 (DEFAULT_TABLE table=default.Table_4c54324167a8434a90e8dcb1354111f9 [id=9da4000ba42645d69f7f3c5fc3a941c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.889235 10370 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet f294831873d548e4b9201c1bbb964096 (table default.Table_de36c9f702024914965bc6b371eda68e [id=e2cdad6dfe7846ebb3c14f5611cf0f18]). Expected version 1 got 0
I20260924 00:21:03.889793 10370 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet 5b0345252c3c496db2987c30adce3da7 (table default.Table_a5d798f7819e403a96f27e98cb0af2bb [id=30a3f7c8fb9c46cf8b883eedae25d938]). Expected version 4 got 0
W20260924 00:21:03.891165 10876 consensus_peers.cc:487] T 5d7d52be4f1f4fc29b8172b9fa878f76 P c8626e300d0644b0908797ad61bd7cef -> Peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Couldn't send request to peer b10a9082a37b46f09d581944914c1670. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5d7d52be4f1f4fc29b8172b9fa878f76. This is attempt 1: this message will repeat every 5th retry.
I20260924 00:21:03.891789 10876 leader_election.cc:302] T 195b033091b543b58ee08417eae2c0e6 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
W20260924 00:21:03.893532 10589 leader_election.cc:341] T 0d555f059f84478eb805f4ca4534dab4 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:03.893921 10589 leader_election.cc:302] T 0d555f059f84478eb805f4ca4534dab4 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:03.895198 11301 raft_consensus.cc:482] T 5b0345252c3c496db2987c30adce3da7 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:03.895355 11308 raft_consensus.cc:2726] T 0d555f059f84478eb805f4ca4534dab4 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:03.895601 11301 raft_consensus.cc:504] T 5b0345252c3c496db2987c30adce3da7 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
W20260924 00:21:03.897311 10589 leader_election.cc:341] T b06bf1e4691441769e162aa5404d4656 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:03.897490 11301 leader_election.cc:288] T 5b0345252c3c496db2987c30adce3da7 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:03.897676 10589 leader_election.cc:302] T b06bf1e4691441769e162aa5404d4656 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:03.892839 10775 tablet_service.cc:1467] Processing CreateTablet for tablet f85ac5b0fd2348d88e1076b781424ab3 (DEFAULT_TABLE table=default.Table_d3c4523a9e0644f7b0ac485b5b4801db [id=ed74607d751e401ea832ec2a9b29982c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.899408 10775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f85ac5b0fd2348d88e1076b781424ab3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.899791 11308 raft_consensus.cc:2726] T b06bf1e4691441769e162aa5404d4656 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:03.903811 10372 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_c5ae901f65d84f1eb74702fe8b71d858"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:03.905247 10372 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c5ae901f65d84f1eb74702fe8b71d858 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:03.907860 10772 tablet_service.cc:1467] Processing CreateTablet for tablet 1d5a76bc420b41098d4ebd3dc59aa1fc (DEFAULT_TABLE table=default.Table_7d1f0174b0c74f7bb29facca7238f8b0 [id=ff77e539ecb44767bb76cf076799a4c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.908618 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b0345252c3c496db2987c30adce3da7" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:03.908895 10772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d5a76bc420b41098d4ebd3dc59aa1fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.909366 10654 raft_consensus.cc:2379] T 5b0345252c3c496db2987c30adce3da7 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48fbd603abe146c585d15196e1b5be08 in current term 1: Already voted for candidate c8626e300d0644b0908797ad61bd7cef in this term.
I20260924 00:21:03.911264 11301 raft_consensus.cc:482] T f294831873d548e4b9201c1bbb964096 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:03.911692 10779 tablet_service.cc:1467] Processing CreateTablet for tablet a53f06276dbb4d9f97a154a8ceb0278d (DEFAULT_TABLE table=default.Table_08bc69c89b564e158a0b6d764972cb19 [id=68d73a1a9f1540809f974b9a21f44c1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.912675 10779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a53f06276dbb4d9f97a154a8ceb0278d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.918556 11322 raft_consensus.cc:482] T ffa011cf8dc446de849bea75a4f20056 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:03.918711 10776 tablet_service.cc:1467] Processing CreateTablet for tablet 2e839fd66fed4472a0892baa9aa85c54 (DEFAULT_TABLE table=default.Table_e724566b223445b7b3988eeb3e811bcd [id=4b08b25314a64fdd8313975f514fb10a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.919019 11322 raft_consensus.cc:504] T ffa011cf8dc446de849bea75a4f20056 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:03.919641 10776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e839fd66fed4472a0892baa9aa85c54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.921514 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa011cf8dc446de849bea75a4f20056" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:03.922174 10369 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_faaa8b08d77d4384a41b2008c9cc98ea" } modify_external_catalogs: true
I20260924 00:21:03.922641 11322 leader_election.cc:288] T ffa011cf8dc446de849bea75a4f20056 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:03.922672 10369 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_faaa8b08d77d4384a41b2008c9cc98ea" } modify_external_catalogs: true
W20260924 00:21:03.923465 10592 leader_election.cc:341] T ffa011cf8dc446de849bea75a4f20056 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:03.925397 10913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98926dd3850d40938b67779d5fd9a1db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.926009 10941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b0345252c3c496db2987c30adce3da7" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:03.927618 10732 leader_election.cc:302] T 5b0345252c3c496db2987c30adce3da7 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08; no voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:03.927795 10941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa011cf8dc446de849bea75a4f20056" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:03.928714 10914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9319f9caadb04067a2e9700b5019ed37. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:21:03.934121 10589 leader_election.cc:341] T ffa011cf8dc446de849bea75a4f20056 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:03.934504 10589 leader_election.cc:302] T ffa011cf8dc446de849bea75a4f20056 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:03.934952 10916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5107b763acf04df88553ba0af2757570. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.935444 11308 raft_consensus.cc:2726] T ffa011cf8dc446de849bea75a4f20056 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:03.937381 10915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 717759eb61674c89b05b8d2b0dc34979. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.941294 10912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c2a729a802e4abe9ab4f989a40198be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.911698 11301 raft_consensus.cc:504] T f294831873d548e4b9201c1bbb964096 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:03.950572 11301 leader_election.cc:288] T f294831873d548e4b9201c1bbb964096 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:03.952039 11301 raft_consensus.cc:3037] T 5b0345252c3c496db2987c30adce3da7 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:03.952003 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f294831873d548e4b9201c1bbb964096" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:03.956204 10636 tablet_service.cc:1467] Processing CreateTablet for tablet 1d5a76bc420b41098d4ebd3dc59aa1fc (DEFAULT_TABLE table=default.Table_7d1f0174b0c74f7bb29facca7238f8b0 [id=ff77e539ecb44767bb76cf076799a4c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.956494 10635 tablet_service.cc:1467] Processing CreateTablet for tablet a53f06276dbb4d9f97a154a8ceb0278d (DEFAULT_TABLE table=default.Table_08bc69c89b564e158a0b6d764972cb19 [id=68d73a1a9f1540809f974b9a21f44c1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.957180 10636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d5a76bc420b41098d4ebd3dc59aa1fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.957122 10629 tablet_service.cc:1467] Processing CreateTablet for tablet f85ac5b0fd2348d88e1076b781424ab3 (DEFAULT_TABLE table=default.Table_d3c4523a9e0644f7b0ac485b5b4801db [id=ed74607d751e401ea832ec2a9b29982c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.958055 10629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f85ac5b0fd2348d88e1076b781424ab3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.958149 10630 tablet_service.cc:1467] Processing CreateTablet for tablet 2e839fd66fed4472a0892baa9aa85c54 (DEFAULT_TABLE table=default.Table_e724566b223445b7b3988eeb3e811bcd [id=4b08b25314a64fdd8313975f514fb10a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.959163 10630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e839fd66fed4472a0892baa9aa85c54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.961313 10635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a53f06276dbb4d9f97a154a8ceb0278d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.963910 10634 tablet_service.cc:1467] Processing CreateTablet for tablet d4e120acff3e4c30aed813b0b24fbfee (DEFAULT_TABLE table=default.Table_faaa8b08d77d4384a41b2008c9cc98ea [id=3fce8b5a6e82492b8a2355183e566142]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.964725 10634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4e120acff3e4c30aed813b0b24fbfee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.985690 10921 tablet_service.cc:1467] Processing CreateTablet for tablet 57ae3d10846a4aaa84eba81b13aafd44 (DEFAULT_TABLE table=default.Table_29e523da31634984b775f31084e82c99 [id=6d2a4bce1272466c8a00217643c0c9f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:03.985909 11254 raft_consensus.cc:2781] T 195b033091b543b58ee08417eae2c0e6 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:03.986356 11254 raft_consensus.cc:482] T 195b033091b543b58ee08417eae2c0e6 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:03.986670 11254 raft_consensus.cc:3037] T 195b033091b543b58ee08417eae2c0e6 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:03.986855 10921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57ae3d10846a4aaa84eba81b13aafd44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:03.989739 10371 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_a743710df3db4dc7af777801a59305e6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:03.989858 10365 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_eff603db4cdf417c871be7eb94681227"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:03.991287 10371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a743710df3db4dc7af777801a59305e6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260924 00:21:03.991482 10365 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eff603db4cdf417c871be7eb94681227 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:03.992559 10367 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_6fd2c4af10ab43ffafa1f24e552df8d3" } new_table_name: "default.Table_6f4df47aea9c41e498e3eb33abc9b832" modify_external_catalogs: true
I20260924 00:21:04.001446 10780 tablet_service.cc:1467] Processing CreateTablet for tablet d4e120acff3e4c30aed813b0b24fbfee (DEFAULT_TABLE table=default.Table_faaa8b08d77d4384a41b2008c9cc98ea [id=3fce8b5a6e82492b8a2355183e566142]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:04.002363 10780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4e120acff3e4c30aed813b0b24fbfee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:04.003391 10773 tablet_service.cc:1467] Processing CreateTablet for tablet 28be3abcaff64a3d97fad866472b41be (DEFAULT_TABLE table=default.Table_a01f8ccbfb7a4cff9315ec54f8f2f50e [id=001b691754074a66b8bbc7fa0f95caff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:04.004465 10773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28be3abcaff64a3d97fad866472b41be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:04.006734 11040 tablet_bootstrap.cc:492] T 59c077eb68794340ac900dbedc7056ee P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:04.007091 11040 ts_tablet_manager.cc:1397] T 59c077eb68794340ac900dbedc7056ee P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.281s user 0.006s sys 0.004s
I20260924 00:21:04.009334 11040 raft_consensus.cc:348] T 59c077eb68794340ac900dbedc7056ee P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:04.012419 11040 raft_consensus.cc:374] T 59c077eb68794340ac900dbedc7056ee P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:04.018503 11049 tablet_replica.cc:330] T d44932f8f08e4ea7acc05b8ec511f045 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:04.035567 10370 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_7d1f0174b0c74f7bb29facca7238f8b0" } new_table_name: "default.Table_efa7000cc59a43639d964cd555a08961" modify_external_catalogs: true
I20260924 00:21:04.044925 10774 tablet_service.cc:1467] Processing CreateTablet for tablet c7cb7c6892404c7fa593cdaaac67f800 (DEFAULT_TABLE table=default.Table_14e49ca823664cc38f51183bc9bdc0ac [id=64a8cee150dd4ad4bba25adf7dbe36e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:04.045046 10373 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_3879c2e59faa453281d3a65092107295"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:04.046056 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7cb7c6892404c7fa593cdaaac67f800. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:21:04.046425 10373 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3879c2e59faa453281d3a65092107295 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:04.051470 11049 raft_consensus.cc:2229] T d44932f8f08e4ea7acc05b8ec511f045 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:04.051658 11301 raft_consensus.cc:2726] T 5b0345252c3c496db2987c30adce3da7 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:04.051928 11049 raft_consensus.cc:2258] T d44932f8f08e4ea7acc05b8ec511f045 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:04.053911 11049 ts_tablet_manager.cc:1905] T d44932f8f08e4ea7acc05b8ec511f045 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:03.997327 11218 consensus_queue.cc:1035] T 5b0345252c3c496db2987c30adce3da7 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Connected to new peer: Peer: permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260924 00:21:04.060366 10359 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.126:34479, user_credentials={real_user=slave}} blocked reactor thread for 109916us
I20260924 00:21:04.072160 11364 tablet.cc:1722] T 85a0ad61b7cb4e11b30e84ddc4655b64 P b10a9082a37b46f09d581944914c1670: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260924 00:21:04.072180 11041 tablet_bootstrap.cc:492] T 0980afc2af174da2a8ca2dd414e9825c P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:04.074702 11041 ts_tablet_manager.cc:1397] T 0980afc2af174da2a8ca2dd414e9825c P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.252s user 0.006s sys 0.003s
I20260924 00:21:04.085867 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 86374d72238d42c29542e649a7a0200d (table default.Table_6a3c73b6e78a4066bb59da236b32ab81 [id=8a99c64662ce4d3bba44aeb578c55c16]) successfully deleted
I20260924 00:21:04.012655 11040 raft_consensus.cc:729] T 59c077eb68794340ac900dbedc7056ee P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:04.110548 10635 tablet_service.cc:1467] Processing CreateTablet for tablet c7cb7c6892404c7fa593cdaaac67f800 (DEFAULT_TABLE table=default.Table_14e49ca823664cc38f51183bc9bdc0ac [id=64a8cee150dd4ad4bba25adf7dbe36e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:04.111503 10635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7cb7c6892404c7fa593cdaaac67f800. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:04.110397 10918 tablet_service.cc:1467] Processing CreateTablet for tablet 1d5a76bc420b41098d4ebd3dc59aa1fc (DEFAULT_TABLE table=default.Table_7d1f0174b0c74f7bb29facca7238f8b0 [id=ff77e539ecb44767bb76cf076799a4c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:04.111877 10906 tablet_service.cc:1467] Processing CreateTablet for tablet c7cb7c6892404c7fa593cdaaac67f800 (DEFAULT_TABLE table=default.Table_14e49ca823664cc38f51183bc9bdc0ac [id=64a8cee150dd4ad4bba25adf7dbe36e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:04.112514 10918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d5a76bc420b41098d4ebd3dc59aa1fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:04.113638 10368 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 717759eb61674c89b05b8d2b0dc34979
I20260924 00:21:04.114387 10366 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 675b9e33edcf4d4ea7fa428b476e3945
I20260924 00:21:04.115454 10366 catalog_manager.cc:6565] ReplaceTablet: tablet 675b9e33edcf4d4ea7fa428b476e3945 deleted and replaced by tablet 915e8072851c4d2b90f8af12b1dc58e0
I20260924 00:21:04.115613 11041 raft_consensus.cc:348] T 0980afc2af174da2a8ca2dd414e9825c P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:04.116248 11041 raft_consensus.cc:374] T 0980afc2af174da2a8ca2dd414e9825c P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:04.116483 11041 raft_consensus.cc:729] T 0980afc2af174da2a8ca2dd414e9825c P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:04.117486 10906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7cb7c6892404c7fa593cdaaac67f800. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:04.118428 10374 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet a0f71049f6b3478da3b07ebc9293ce64
I20260924 00:21:04.117334 11341 consensus_queue.cc:1035] T fcf651adb1d3459facfe3a8df2a91ed7 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Connected to new peer: Peer: permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260924 00:21:04.119601 10374 catalog_manager.cc:6565] ReplaceTablet: tablet a0f71049f6b3478da3b07ebc9293ce64 deleted and replaced by tablet d574642ebc6340fcb5e83c36804ee9f8
I20260924 00:21:04.120044 10369 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet d4e120acff3e4c30aed813b0b24fbfee
I20260924 00:21:04.120918 11041 consensus_queue.cc:260] T 0980afc2af174da2a8ca2dd414e9825c P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:04.128675 10368 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 96960f3061fb43e9939b470f49edf1e9 (table default.Table_477c0f1cce9a4c089551e4e767ba7ae0 [id=6251651eb5944ee086dd564d86204148]). Expected version 2 got 0
I20260924 00:21:04.131757 10634 tablet_service.cc:1467] Processing CreateTablet for tablet 28be3abcaff64a3d97fad866472b41be (DEFAULT_TABLE table=default.Table_a01f8ccbfb7a4cff9315ec54f8f2f50e [id=001b691754074a66b8bbc7fa0f95caff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:04.132714 10634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28be3abcaff64a3d97fad866472b41be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:04.104144 11366 tablet.cc:1722] T f294831873d548e4b9201c1bbb964096 P b10a9082a37b46f09d581944914c1670: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260924 00:21:04.146153 10920 tablet_service.cc:1467] Processing CreateTablet for tablet f85ac5b0fd2348d88e1076b781424ab3 (DEFAULT_TABLE table=default.Table_d3c4523a9e0644f7b0ac485b5b4801db [id=ed74607d751e401ea832ec2a9b29982c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:04.147272 10920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f85ac5b0fd2348d88e1076b781424ab3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:04.149142 10919 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:35358 (request call id 140) took 1003 ms. Trace:
I20260924 00:21:04.149508 10919 rpcz_store.cc:276] 0924 00:21:03.107762 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:03.112562 (+ 4800us) service_pool.cc:227] Handling call
0924 00:21:03.115419 (+ 2857us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 00:21:03.115498 (+ 79us) ts_tablet_manager.cc:676] Creating new metadata...
0924 00:21:03.499120 (+383622us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:04.082094 (+582974us) tablet_replica.cc:205] Creating consensus instance
0924 00:21:04.111273 (+ 29179us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":152351}
I20260924 00:21:04.150738 10922 tablet_service.cc:1514] Processing DeleteTablet for tablet cd30f52583124543b5d616ef94b31831 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:02 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:04.151319 11051 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35358 (request call id 138) took 1394 ms. Trace:
I20260924 00:21:04.151605 11051 rpcz_store.cc:276] 0924 00:21:02.723438 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:02.723539 (+ 101us) service_pool.cc:227] Handling call
0924 00:21:03.143546 (+420007us) ts_tablet_manager.cc:1145] Deleting tablet 7463fcb18e884b1dbc478ebaa19f6cb4
0924 00:21:03.143589 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:03.387568 (+243979us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:03.668177 (+280609us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:04.113224 (+445047us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":302921,"tablet-delete.queue_time_us":419522}
I20260924 00:21:04.152894 11051 tablet_replica.cc:330] T 5d7d52be4f1f4fc29b8172b9fa878f76 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:04.168618 11071 server_base.cc:1029] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
W20260924 00:20:57.780687 11127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:21:04.168768 11130 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection time-out
W20260924 00:20:57.764353 11132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:21:04.174489 10908 tablet_service.cc:1467] Processing CreateTablet for tablet d4e120acff3e4c30aed813b0b24fbfee (DEFAULT_TABLE table=default.Table_faaa8b08d77d4384a41b2008c9cc98ea [id=3fce8b5a6e82492b8a2355183e566142]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:04.175586 10908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4e120acff3e4c30aed813b0b24fbfee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:04.176599 11071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:21:04.177721 10780 tablet_service.cc:1514] Processing DeleteTablet for tablet a0f71049f6b3478da3b07ebc9293ce64 with delete_type TABLET_DATA_DELETED (replaced by tablet d574642ebc6340fcb5e83c36804ee9f8) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:04.177956 10778 tablet_service.cc:1514] Processing DeleteTablet for tablet d4e120acff3e4c30aed813b0b24fbfee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:03 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:04.180087 10777 tablet_service.cc:1467] Processing CreateTablet for tablet d32a0c74b527451fb723e3851d5d055b (DEFAULT_TABLE table=default.Table_0dc486ba6e6d4091912ecb311b5dbd33 [id=a3f51096ae8b4b3188795babd04bd547]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:04.181160 10776 tablet_service.cc:1514] Processing DeleteTablet for tablet 675b9e33edcf4d4ea7fa428b476e3945 with delete_type TABLET_DATA_DELETED (replaced by tablet 915e8072851c4d2b90f8af12b1dc58e0) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:04.181825 10910 tablet_service.cc:1467] Processing CreateTablet for tablet a53f06276dbb4d9f97a154a8ceb0278d (DEFAULT_TABLE table=default.Table_08bc69c89b564e158a0b6d764972cb19 [id=68d73a1a9f1540809f974b9a21f44c1b]), partition=RANGE (key) PARTITION UNBOUNDED
W20260924 00:21:04.182770 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet a0f71049f6b3478da3b07ebc9293ce64 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a0f71049f6b3478da3b07ebc9293ce64 already in progress: creating tablet
I20260924 00:21:04.183982 10910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a53f06276dbb4d9f97a154a8ceb0278d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:04.186995 11071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:21:04.188422 11071 hybrid_clock.cc:648] HybridClock initialized: now 1790209264188367 us; error 45 us; skew 500 ppm
I20260924 00:21:04.189625 11071 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 00:21:04.110911 11040 consensus_queue.cc:260] T 59c077eb68794340ac900dbedc7056ee P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:04.181952 10775 tablet_service.cc:1514] Processing DeleteTablet for tablet 717759eb61674c89b05b8d2b0dc34979 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:03 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:04.197520 10909 tablet_service.cc:1467] Processing CreateTablet for tablet 2e839fd66fed4472a0892baa9aa85c54 (DEFAULT_TABLE table=default.Table_e724566b223445b7b3988eeb3e811bcd [id=4b08b25314a64fdd8313975f514fb10a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:04.204221 10909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e839fd66fed4472a0892baa9aa85c54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:04.208788 11071 webserver.cc:460] Webserver started at http://127.10.4.124:35831/ using document root <none> and password file <none>
I20260924 00:21:04.210067 11071 fs_manager.cc:359] Metadata directory not provided
I20260924 00:21:04.210362 11071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:21:04.208012 10907 tablet_service.cc:1467] Processing CreateTablet for tablet 28be3abcaff64a3d97fad866472b41be (DEFAULT_TABLE table=default.Table_a01f8ccbfb7a4cff9315ec54f8f2f50e [id=001b691754074a66b8bbc7fa0f95caff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:04.213045 10907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28be3abcaff64a3d97fad866472b41be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:04.219317 10777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d32a0c74b527451fb723e3851d5d055b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:21:04.227013 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet d4e120acff3e4c30aed813b0b24fbfee because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d4e120acff3e4c30aed813b0b24fbfee already in progress: creating tablet
W20260924 00:21:04.227639 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 675b9e33edcf4d4ea7fa428b476e3945 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 675b9e33edcf4d4ea7fa428b476e3945 already in progress: creating tablet
W20260924 00:21:04.228163 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 717759eb61674c89b05b8d2b0dc34979 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 717759eb61674c89b05b8d2b0dc34979 already in progress: creating tablet
W20260924 00:21:04.229434 10370 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1d5a76bc420b41098d4ebd3dc59aa1fc consensus state has no leader
W20260924 00:21:04.231941 10367 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4e9f00abffe14046bf47a6cae7010690 consensus state has no leader
I20260924 00:21:04.232124 11365 tablet.cc:1722] T 85a0ad61b7cb4e11b30e84ddc4655b64 P 48fbd603abe146c585d15196e1b5be08: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260924 00:21:04.259397 11039 tablet_bootstrap.cc:492] T f77aea0aad5a430f941aaec0a1c19640 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:04.259855 11039 ts_tablet_manager.cc:1397] T f77aea0aad5a430f941aaec0a1c19640 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.936s user 0.011s sys 0.012s
I20260924 00:21:04.263482 11039 raft_consensus.cc:348] T f77aea0aad5a430f941aaec0a1c19640 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
W20260924 00:21:04.270458 10732 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:41301, user_credentials={real_user=slave}} blocked reactor thread for 342911us
I20260924 00:21:04.272388 10939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f294831873d548e4b9201c1bbb964096" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:04.265082 11039 raft_consensus.cc:374] T f77aea0aad5a430f941aaec0a1c19640 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:04.274981 11040 ts_tablet_manager.cc:1428] T 59c077eb68794340ac900dbedc7056ee P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.268s user 0.004s sys 0.004s
I20260924 00:21:04.275014 11039 raft_consensus.cc:729] T f77aea0aad5a430f941aaec0a1c19640 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:04.276930 11040 tablet_bootstrap.cc:492] T 666755ec87bb496db75e43ec9204ec5b P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:04.286788 11049 ts_tablet_manager.cc:1918] T d44932f8f08e4ea7acc05b8ec511f045 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:04.288246 11049 log.cc:1196] T d44932f8f08e4ea7acc05b8ec511f045 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/d44932f8f08e4ea7acc05b8ec511f045
I20260924 00:21:04.288990 10732 leader_election.cc:302] T f294831873d548e4b9201c1bbb964096 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08; no voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:04.290766 11051 raft_consensus.cc:2229] T 5d7d52be4f1f4fc29b8172b9fa878f76 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:21:04.290898 11301 raft_consensus.cc:2726] T f294831873d548e4b9201c1bbb964096 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:04.291318 11051 pending_rounds.cc:62] T 5d7d52be4f1f4fc29b8172b9fa878f76 P c8626e300d0644b0908797ad61bd7cef: Trying to abort 1 pending ops.
I20260924 00:21:04.291507 11051 pending_rounds.cc:68] T 5d7d52be4f1f4fc29b8172b9fa878f76 P c8626e300d0644b0908797ad61bd7cef: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332697139054858240 op_type: NO_OP noop_request { }
I20260924 00:21:04.291909 11051 raft_consensus.cc:2866] T 5d7d52be4f1f4fc29b8172b9fa878f76 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260924 00:21:04.291921 10773 tablet_service.cc:1514] Processing DeleteTablet for tablet 59c077eb68794340ac900dbedc7056ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:02 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:04.292186 11051 raft_consensus.cc:2258] T 5d7d52be4f1f4fc29b8172b9fa878f76 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:04.292586 11335 tablet_replica.cc:330] T 59c077eb68794340ac900dbedc7056ee P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:04.292810 11049 ts_tablet_manager.cc:1939] T d44932f8f08e4ea7acc05b8ec511f045 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:04.293344 11335 raft_consensus.cc:2229] T 59c077eb68794340ac900dbedc7056ee P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:04.293789 11335 raft_consensus.cc:2258] T 59c077eb68794340ac900dbedc7056ee P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:04.294358 11051 ts_tablet_manager.cc:1905] T 5d7d52be4f1f4fc29b8172b9fa878f76 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:04.307236 11049 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 129) took 1852 ms. Trace:
I20260924 00:21:04.307561 11049 rpcz_store.cc:276] 0924 00:21:02.454442 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:02.454576 (+ 134us) service_pool.cc:227] Handling call
0924 00:21:04.018339 (+1563763us) ts_tablet_manager.cc:1145] Deleting tablet d44932f8f08e4ea7acc05b8ec511f045
0924 00:21:04.018396 (+ 57us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:04.168302 (+149906us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:04.286541 (+118239us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:04.307165 (+ 20624us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1563239}
I20260924 00:21:04.308568 11049 tablet_replica.cc:330] T 7463fcb18e884b1dbc478ebaa19f6cb4 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:04.309353 11049 raft_consensus.cc:2229] T 7463fcb18e884b1dbc478ebaa19f6cb4 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:04.310832 11308 raft_consensus.cc:482] T 0980afc2af174da2a8ca2dd414e9825c P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:04.311223 11308 raft_consensus.cc:504] T 0980afc2af174da2a8ca2dd414e9825c P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:04.313580 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet d44932f8f08e4ea7acc05b8ec511f045 (table default.Table_321e6959ada84ccea6413114309bc3b8 [id=8092ca5df917480aba7e26a4cedf9fbd]) successfully deleted
I20260924 00:21:04.314291 10633 tablet_service.cc:1514] Processing DeleteTablet for tablet d4e120acff3e4c30aed813b0b24fbfee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:03 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:04.316352 10625 tablet_service.cc:1467] Processing CreateTablet for tablet d32a0c74b527451fb723e3851d5d055b (DEFAULT_TABLE table=default.Table_0dc486ba6e6d4091912ecb311b5dbd33 [id=a3f51096ae8b4b3188795babd04bd547]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:04.317973 10627 tablet_service.cc:1514] Processing DeleteTablet for tablet b06bf1e4691441769e162aa5404d4656 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:59 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:04.319281 10628 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d555f059f84478eb805f4ca4534dab4 with delete_type TABLET_DATA_DELETED (Sending initial creation of tablet) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:04.317469 10630 tablet_service.cc:1514] Processing DeleteTablet for tablet 717759eb61674c89b05b8d2b0dc34979 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:03 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:04.319660 10373 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_6496e9c31f6349e78e6f469660e561d6" } modify_external_catalogs: true
I20260924 00:21:04.320070 11308 leader_election.cc:288] T 0980afc2af174da2a8ca2dd414e9825c P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:04.320158 10373 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_6496e9c31f6349e78e6f469660e561d6" } modify_external_catalogs: true
I20260924 00:21:04.318550 10631 tablet_service.cc:1514] Processing DeleteTablet for tablet a0f71049f6b3478da3b07ebc9293ce64 with delete_type TABLET_DATA_DELETED (replaced by tablet d574642ebc6340fcb5e83c36804ee9f8) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:04.317567 10632 tablet_service.cc:1514] Processing DeleteTablet for tablet 675b9e33edcf4d4ea7fa428b476e3945 with delete_type TABLET_DATA_DELETED (replaced by tablet 915e8072851c4d2b90f8af12b1dc58e0) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:04.325167 11335 ts_tablet_manager.cc:1905] T 59c077eb68794340ac900dbedc7056ee P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:04.325855 10371 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_4c54324167a8434a90e8dcb1354111f9" } modify_external_catalogs: true
I20260924 00:21:04.326320 10371 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_4c54324167a8434a90e8dcb1354111f9" } modify_external_catalogs: true
I20260924 00:21:04.330065 10367 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_d3365a0130ed438d9a05029476376802"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:04.331449 10367 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d3365a0130ed438d9a05029476376802 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:04.309759 11049 raft_consensus.cc:2258] T 7463fcb18e884b1dbc478ebaa19f6cb4 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:04.340080 11040 tablet_bootstrap.cc:654] T 666755ec87bb496db75e43ec9204ec5b P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:04.343904 11049 ts_tablet_manager.cc:1905] T 7463fcb18e884b1dbc478ebaa19f6cb4 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:04.353618 11364 tablet.cc:1722] T 96960f3061fb43e9939b470f49edf1e9 P b10a9082a37b46f09d581944914c1670: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260924 00:21:04.368074 10941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0980afc2af174da2a8ca2dd414e9825c" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
W20260924 00:21:04.370806 10589 leader_election.cc:341] T 0980afc2af174da2a8ca2dd414e9825c P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:04.375516 10371 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 9319f9caadb04067a2e9700b5019ed37
I20260924 00:21:04.382633 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0980afc2af174da2a8ca2dd414e9825c" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:04.385506 10592 leader_election.cc:341] T 0980afc2af174da2a8ca2dd414e9825c P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:04.385340 11341 consensus_queue.cc:1035] T 59c077eb68794340ac900dbedc7056ee P c8626e300d0644b0908797ad61bd7cef [LEADER]: Connected to new peer: Peer: permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:04.386327 10592 leader_election.cc:302] T 0980afc2af174da2a8ca2dd414e9825c P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:04.390568 11254 raft_consensus.cc:504] T 195b033091b543b58ee08417eae2c0e6 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:04.393904 10626 tablet_service.cc:1514] Processing DeleteTablet for tablet 9319f9caadb04067a2e9700b5019ed37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:04 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:04.394235 11041 ts_tablet_manager.cc:1428] T 0980afc2af174da2a8ca2dd414e9825c P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.297s user 0.000s sys 0.006s
I20260924 00:21:04.397392 10917 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:35358 (request call id 143) took 1010 ms. Trace:
I20260924 00:21:04.397763 10917 rpcz_store.cc:276] 0924 00:21:03.250197 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:03.250756 (+ 559us) service_pool.cc:227] Handling call
0924 00:21:03.261695 (+ 10939us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 00:21:03.309691 (+ 47996us) ts_tablet_manager.cc:676] Creating new metadata...
0924 00:21:04.119165 (+809474us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:04.141833 (+ 22668us) tablet_replica.cc:205] Creating consensus instance
0924 00:21:04.260670 (+118837us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 00:21:04.398129 11308 raft_consensus.cc:2726] T 0980afc2af174da2a8ca2dd414e9825c P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:04.398612 10365 catalog_manager.cc:6040] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: default.Table_67493ad72aac4bbca26a6a322a60e5cd [id=12285914811f4220822b47a2a62b01ab] alter complete (version 2)
I20260924 00:21:04.285988 11039 consensus_queue.cc:260] T f77aea0aad5a430f941aaec0a1c19640 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:04.406692 11365 tablet.cc:1722] T 96960f3061fb43e9939b470f49edf1e9 P 48fbd603abe146c585d15196e1b5be08: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260924 00:21:04.409878 10656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "195b033091b543b58ee08417eae2c0e6" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:21:04.410494 10656 raft_consensus.cc:3037] T 195b033091b543b58ee08417eae2c0e6 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:04.421430 10656 raft_consensus.cc:2454] T 195b033091b543b58ee08417eae2c0e6 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 1.
I20260924 00:21:04.426626 10625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d32a0c74b527451fb723e3851d5d055b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:04.432641 10371 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_c5ae901f65d84f1eb74702fe8b71d858" } new_table_name: "default.Table_82ad1ad5194e43008bb85e5b99ae0d44" modify_external_catalogs: true
I20260924 00:21:04.437238 10367 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_0c36f6e4aa264a4394fd0bc65fd035a8" } new_table_name: "default.Table_9172ff8a00a54ed581e96de0f0b44c3c" modify_external_catalogs: true
I20260924 00:21:04.439010 10370 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_54a305f538ab4d24811b19d3e44e4a90"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:04.440339 10370 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_54a305f538ab4d24811b19d3e44e4a90 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:04.442956 11040 tablet_bootstrap.cc:492] T 666755ec87bb496db75e43ec9204ec5b P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:04.443320 11040 ts_tablet_manager.cc:1397] T 666755ec87bb496db75e43ec9204ec5b P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.167s user 0.007s sys 0.003s
I20260924 00:21:04.444180 10368 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_825c6492de4d43f7ac0521a88ab58441"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:04.445566 10368 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_825c6492de4d43f7ac0521a88ab58441 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:04.445482 11040 raft_consensus.cc:348] T 666755ec87bb496db75e43ec9204ec5b P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:04.452093 11040 raft_consensus.cc:374] T 666755ec87bb496db75e43ec9204ec5b P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:04.452378 11040 raft_consensus.cc:729] T 666755ec87bb496db75e43ec9204ec5b P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:04.453087 11040 consensus_queue.cc:260] T 666755ec87bb496db75e43ec9204ec5b P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:04.456660 10372 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_6ab0fc6ae11943bea05a803069f3da86"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:04.458010 10372 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6ab0fc6ae11943bea05a803069f3da86 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260924 00:21:04.459368 10367 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cb51d5c6460049108cd7e1974821e578 consensus state has no leader
W20260924 00:21:04.462502 10371 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 84e73180fab74524b4d568a010dc7175 has no consensus state
I20260924 00:21:04.480141 11041 tablet_bootstrap.cc:492] T 2964cd68d5e94f3780a6a3157bd86983 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:04.440860 11335 ts_tablet_manager.cc:1918] T 59c077eb68794340ac900dbedc7056ee P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:04.483646 11335 log.cc:1196] T 59c077eb68794340ac900dbedc7056ee P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/59c077eb68794340ac900dbedc7056ee
I20260924 00:21:04.488533 11335 ts_tablet_manager.cc:1939] T 59c077eb68794340ac900dbedc7056ee P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:04.492015 11384 tablet.cc:1722] T 96960f3061fb43e9939b470f49edf1e9 P c8626e300d0644b0908797ad61bd7cef: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260924 00:21:04.485720 11040 ts_tablet_manager.cc:1428] T 666755ec87bb496db75e43ec9204ec5b P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.042s user 0.006s sys 0.000s
I20260924 00:21:04.497189 11040 tablet_bootstrap.cc:492] T fcf651adb1d3459facfe3a8df2a91ed7 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:04.510006 11040 tablet_bootstrap.cc:654] T fcf651adb1d3459facfe3a8df2a91ed7 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:04.512465 11041 tablet_bootstrap.cc:654] T 2964cd68d5e94f3780a6a3157bd86983 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:04.509557 11049 ts_tablet_manager.cc:1918] T 7463fcb18e884b1dbc478ebaa19f6cb4 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:04.513602 11049 log.cc:1196] T 7463fcb18e884b1dbc478ebaa19f6cb4 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/7463fcb18e884b1dbc478ebaa19f6cb4
I20260924 00:21:04.515264 11049 ts_tablet_manager.cc:1939] T 7463fcb18e884b1dbc478ebaa19f6cb4 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:04.517405 11049 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 138) took 1802 ms. Trace:
I20260924 00:21:04.521013 11049 rpcz_store.cc:276] 0924 00:21:02.714652 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:02.714786 (+ 134us) service_pool.cc:227] Handling call
0924 00:21:04.308423 (+1593637us) ts_tablet_manager.cc:1145] Deleting tablet 7463fcb18e884b1dbc478ebaa19f6cb4
0924 00:21:04.308466 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:04.404520 (+ 96054us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:04.509344 (+104824us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:04.517238 (+ 7894us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1559194}
I20260924 00:21:04.523455 11039 ts_tablet_manager.cc:1428] T f77aea0aad5a430f941aaec0a1c19640 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.263s user 0.004s sys 0.002s
I20260924 00:21:04.524331 11039 tablet_bootstrap.cc:492] T 772fb3d6180e499daf94ceeb9b26048d P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:04.524919 11049 tablet_replica.cc:330] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:04.526859 11049 raft_consensus.cc:2229] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:04.527343 11049 raft_consensus.cc:2258] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:04.534649 11254 leader_election.cc:288] T 195b033091b543b58ee08417eae2c0e6 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:21:04.538429 11049 ts_tablet_manager.cc:1905] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:04.547683 10876 leader_election.cc:302] T 195b033091b543b58ee08417eae2c0e6 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:04.549134 11341 raft_consensus.cc:2781] T 195b033091b543b58ee08417eae2c0e6 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:04.549532 11341 raft_consensus.cc:686] T 195b033091b543b58ee08417eae2c0e6 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Becoming Leader. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Running, Role: LEADER
I20260924 00:21:04.550164 10367 master_service.cc:946] ReplaceTablet: received request to replace tablet 0c583d9ba1414f8ea2e605b4b6b076f8 from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:04.550285 11341 consensus_queue.cc:237] T 195b033091b543b58ee08417eae2c0e6 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:04.588907 10374 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_0dc486ba6e6d4091912ecb311b5dbd33" } modify_external_catalogs: true
I20260924 00:21:04.589459 10374 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_0dc486ba6e6d4091912ecb311b5dbd33" } modify_external_catalogs: true
W20260924 00:21:04.594976 10359 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.126:34479, user_credentials={real_user=slave}} blocked reactor thread for 115265us
I20260924 00:21:04.595603 11368 tablet.cc:1717] T 96960f3061fb43e9939b470f49edf1e9 P b10a9082a37b46f09d581944914c1670: Skipping requested alter to schema version 2, tablet already version 2
I20260924 00:21:04.597697 11365 tablet.cc:1717] T 96960f3061fb43e9939b470f49edf1e9 P 48fbd603abe146c585d15196e1b5be08: Skipping requested alter to schema version 2, tablet already version 2
I20260924 00:21:04.579684 10369 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_e3fed8d230874b52bf545da40daf0c48"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:04.595589 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 7463fcb18e884b1dbc478ebaa19f6cb4 (table default.Table_d01ccf505bd94805b62c8f0755dad1d1 [id=8e2fe68fe5da4414b7269f526b8af683]) successfully deleted
W20260924 00:21:04.604038 10369 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e3fed8d230874b52bf545da40daf0c48 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:04.618165 11071 fs_manager.cc:709] Time spent opening directory manager: real 0.367s user 0.045s sys 0.048s
I20260924 00:21:04.621613 10370 master_service.cc:946] ReplaceTablet: received request to replace tablet 5107b763acf04df88553ba0af2757570 from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:04.624166 11041 tablet_bootstrap.cc:492] T 2964cd68d5e94f3780a6a3157bd86983 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:04.624521 11041 ts_tablet_manager.cc:1397] T 2964cd68d5e94f3780a6a3157bd86983 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.172s user 0.008s sys 0.010s
I20260924 00:21:04.626612 11041 raft_consensus.cc:348] T 2964cd68d5e94f3780a6a3157bd86983 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:04.627352 11041 raft_consensus.cc:374] T 2964cd68d5e94f3780a6a3157bd86983 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:04.627588 11041 raft_consensus.cc:729] T 2964cd68d5e94f3780a6a3157bd86983 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:04.628274 11041 consensus_queue.cc:260] T 2964cd68d5e94f3780a6a3157bd86983 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:04.632154 11041 ts_tablet_manager.cc:1428] T 2964cd68d5e94f3780a6a3157bd86983 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20260924 00:21:04.634410 11041 tablet_bootstrap.cc:492] T f3432ac96da94d5099bcf9764c2b32e1 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:04.640331 11041 tablet_bootstrap.cc:654] T f3432ac96da94d5099bcf9764c2b32e1 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:04.646482 11049 ts_tablet_manager.cc:1918] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:04.646859 11049 log.cc:1196] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/c7e50ac8b7f947b1bfba8cd1cff0ea7e
I20260924 00:21:04.656826 11049 ts_tablet_manager.cc:1939] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:04.658202 11393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:21:04.658902 11049 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 140) took 1844 ms. Trace:
I20260924 00:21:04.662678 10372 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_08bc69c89b564e158a0b6d764972cb19" } new_table_name: "default.Table_0179ec39d29c4bad9ed0def94394be47" modify_external_catalogs: true
I20260924 00:21:04.662947 11071 fs_manager.cc:725] Time spent opening block manager: real 0.022s user 0.004s sys 0.002s
W20260924 00:21:04.664319 10358 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet cb51d5c6460049108cd7e1974821e578 (table default.Table_9172ff8a00a54ed581e96de0f0b44c3c [id=ed975cbe1bb04e3194882bed0f4ec0b1], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet cb51d5c6460049108cd7e1974821e578 consensus state has no leader
I20260924 00:21:04.666437 10368 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_7b8c1651b4974bd5b74177dddadf5c12" } new_table_name: "default.Table_c960d2a8d86a4ce3ba1dbd2f57d3d8b7" modify_external_catalogs: true
I20260924 00:21:04.659211 11049 rpcz_store.cc:276] 0924 00:21:02.814706 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:02.814805 (+ 99us) service_pool.cc:227] Handling call
0924 00:21:04.524785 (+1709980us) ts_tablet_manager.cc:1145] Deleting tablet c7e50ac8b7f947b1bfba8cd1cff0ea7e
0924 00:21:04.524821 (+ 36us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:04.555343 (+ 30522us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:04.646236 (+ 90893us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:04.658837 (+ 12601us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1709465}
I20260924 00:21:04.672088 11049 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 139) took 1858 ms. Trace:
I20260924 00:21:04.672637 10371 catalog_manager.cc:6040] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: default.Table_477c0f1cce9a4c089551e4e767ba7ae0 [id=6251651eb5944ee086dd564d86204148] alter complete (version 2)
I20260924 00:21:04.672446 11049 rpcz_store.cc:276] 0924 00:21:02.814060 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:02.815594 (+ 1534us) service_pool.cc:227] Handling call
0924 00:21:04.671835 (+1856241us) ts_tablet_manager.cc:1145] Deleting tablet ccb26e0fbc1948bc9a00e9e26c4d1139
0924 00:21:04.671873 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:04.672032 (+ 159us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1855591}
I20260924 00:21:04.673602 11049 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 143) took 1499 ms. Trace:
I20260924 00:21:04.673910 11049 rpcz_store.cc:276] 0924 00:21:03.173771 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:03.173899 (+ 128us) service_pool.cc:227] Handling call
0924 00:21:04.673267 (+1499368us) ts_tablet_manager.cc:1145] Deleting tablet 59a0a950a8144e99b2ed47be9a567d9a
0924 00:21:04.673303 (+ 36us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:04.673553 (+ 250us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1498564}
I20260924 00:21:04.674827 11049 tablet_replica.cc:330] T 59c077eb68794340ac900dbedc7056ee P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:04.675606 11049 raft_consensus.cc:2229] T 59c077eb68794340ac900dbedc7056ee P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:04.676028 11049 pending_rounds.cc:62] T 59c077eb68794340ac900dbedc7056ee P b10a9082a37b46f09d581944914c1670: Trying to abort 1 pending ops.
I20260924 00:21:04.676229 11049 pending_rounds.cc:68] T 59c077eb68794340ac900dbedc7056ee P b10a9082a37b46f09d581944914c1670: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332697141977079808 op_type: NO_OP noop_request { }
I20260924 00:21:04.676610 11049 raft_consensus.cc:2866] T 59c077eb68794340ac900dbedc7056ee P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260924 00:21:04.663326 11071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-2/data,/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-2/wal
uuid: "f8bd0b3184f84f5f8da10c9c21472f92"
format_stamp: "Formatted at 2026-09-24 00:20:47 on dist-test-slave-g3nd"
I20260924 00:21:04.682986 11071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:21:04.672081 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet c7e50ac8b7f947b1bfba8cd1cff0ea7e (table default.Table_593b4d43f8144c04ab048e8221b3fc8e [id=951c867d04f74728a684d20d1104c054]) successfully deleted
W20260924 00:21:04.692190 10361 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.124:35345, user_credentials={real_user=slave}} blocked reactor thread for 385260us
I20260924 00:21:04.693732 11049 raft_consensus.cc:2258] T 59c077eb68794340ac900dbedc7056ee P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:04.618678 11051 ts_tablet_manager.cc:1918] T 5d7d52be4f1f4fc29b8172b9fa878f76 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:04.703559 11049 ts_tablet_manager.cc:1905] T 59c077eb68794340ac900dbedc7056ee P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260924 00:21:04.704466 11371 log.cc:924] Time spent T 85a0ad61b7cb4e11b30e84ddc4655b64 P c8626e300d0644b0908797ad61bd7cef: Append to log took a long time: real 0.193s user 0.000s sys 0.000s
I20260924 00:21:04.707890 11256 raft_consensus.cc:482] T f77aea0aad5a430f941aaec0a1c19640 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:04.708292 11256 raft_consensus.cc:504] T f77aea0aad5a430f941aaec0a1c19640 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:04.709460 11040 tablet_bootstrap.cc:492] T fcf651adb1d3459facfe3a8df2a91ed7 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:04.709824 11040 ts_tablet_manager.cc:1397] T fcf651adb1d3459facfe3a8df2a91ed7 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.213s user 0.026s sys 0.027s
I20260924 00:21:04.710042 11256 leader_election.cc:288] T f77aea0aad5a430f941aaec0a1c19640 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:04.722718 11040 raft_consensus.cc:348] T fcf651adb1d3459facfe3a8df2a91ed7 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:04.723412 11040 raft_consensus.cc:374] T fcf651adb1d3459facfe3a8df2a91ed7 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:04.723662 11040 raft_consensus.cc:729] T fcf651adb1d3459facfe3a8df2a91ed7 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:04.724296 11040 consensus_queue.cc:260] T fcf651adb1d3459facfe3a8df2a91ed7 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:04.729082 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "666755ec87bb496db75e43ec9204ec5b" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:04.729635 10797 raft_consensus.cc:2454] T 666755ec87bb496db75e43ec9204ec5b P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
I20260924 00:21:04.730437 10796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "195b033091b543b58ee08417eae2c0e6" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:04.692642 10359 catalog_manager.cc:4808] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet ccb26e0fbc1948bc9a00e9e26c4d1139 because the tablet was not found. No further retry: Not found: Tablet not found: ccb26e0fbc1948bc9a00e9e26c4d1139
I20260924 00:21:04.732471 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 7463fcb18e884b1dbc478ebaa19f6cb4 (table default.Table_d01ccf505bd94805b62c8f0755dad1d1 [id=8e2fe68fe5da4414b7269f526b8af683]) successfully deleted
I20260924 00:21:04.734539 10777 tablet_service.cc:1514] Processing DeleteTablet for tablet 9319f9caadb04067a2e9700b5019ed37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:04 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:04.735067 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 59c077eb68794340ac900dbedc7056ee (table default.Table_5cf772cb831f4e7e97430617b962db80 [id=226555984f9741d3955c34512885c9cc]) successfully deleted
I20260924 00:21:04.701922 11051 log.cc:1196] T 5d7d52be4f1f4fc29b8172b9fa878f76 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/5d7d52be4f1f4fc29b8172b9fa878f76
I20260924 00:21:04.736250 10777 tablet_service.cc:1514] Processing DeleteTablet for tablet 666755ec87bb496db75e43ec9204ec5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:01 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:04.736604 11051 ts_tablet_manager.cc:1939] T 5d7d52be4f1f4fc29b8172b9fa878f76 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:04.736893 11335 tablet_replica.cc:330] T 666755ec87bb496db75e43ec9204ec5b P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:04.737644 11335 raft_consensus.cc:2229] T 666755ec87bb496db75e43ec9204ec5b P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:04.738054 11335 raft_consensus.cc:2258] T 666755ec87bb496db75e43ec9204ec5b P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:04.738610 11051 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35358 (request call id 132) took 2133 ms. Trace:
I20260924 00:21:04.738884 11051 rpcz_store.cc:276] 0924 00:21:02.605190 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:02.660639 (+ 55449us) service_pool.cc:227] Handling call
0924 00:21:04.152729 (+1492090us) ts_tablet_manager.cc:1145] Deleting tablet 5d7d52be4f1f4fc29b8172b9fa878f76
0924 00:21:04.152787 (+ 58us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:04.483985 (+331198us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:04.509555 (+ 25570us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:04.738550 (+228995us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":137123,"tablet-delete.queue_time_us":1422106}
I20260924 00:21:04.740084 11335 ts_tablet_manager.cc:1905] T 666755ec87bb496db75e43ec9204ec5b P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:04.740089 11051 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35358 (request call id 142) took 1578 ms. Trace:
I20260924 00:21:04.740337 11051 rpcz_store.cc:276] 0924 00:21:03.161325 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:03.178201 (+ 16876us) service_pool.cc:227] Handling call
0924 00:21:04.739748 (+1561547us) ts_tablet_manager.cc:1145] Deleting tablet 59a0a950a8144e99b2ed47be9a567d9a
0924 00:21:04.739790 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:04.740043 (+ 253us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1488311}
I20260924 00:21:04.744560 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "195b033091b543b58ee08417eae2c0e6" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:21:04.745651 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f77aea0aad5a430f941aaec0a1c19640" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:04.746048 11040 ts_tablet_manager.cc:1428] T fcf651adb1d3459facfe3a8df2a91ed7 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.036s user 0.002s sys 0.003s
I20260924 00:21:04.748805 11051 tablet_replica.cc:330] T 59c077eb68794340ac900dbedc7056ee P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:04.749423 11040 tablet_bootstrap.cc:492] T 7463fcb18e884b1dbc478ebaa19f6cb4 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
W20260924 00:21:04.747889 10876 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:33461, user_credentials={real_user=slave}} blocked reactor thread for 44165.6us
W20260924 00:21:04.771728 10372 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a53f06276dbb4d9f97a154a8ceb0278d consensus state has no leader
W20260924 00:21:04.772787 10368 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 72d3bb80e1034881b94eb75de5d900c2 consensus state has no leader
W20260924 00:21:04.773195 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 9319f9caadb04067a2e9700b5019ed37 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9319f9caadb04067a2e9700b5019ed37 already in progress: creating tablet
I20260924 00:21:04.773599 11040 tablet_bootstrap.cc:654] T 7463fcb18e884b1dbc478ebaa19f6cb4 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:04.786752 11049 ts_tablet_manager.cc:1918] T 59c077eb68794340ac900dbedc7056ee P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:04.751871 10912 tablet_service.cc:1514] Processing DeleteTablet for tablet 717759eb61674c89b05b8d2b0dc34979 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:03 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:04.799696 11039 tablet_bootstrap.cc:654] T 772fb3d6180e499daf94ceeb9b26048d P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:04.802225 10368 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_284132ac952a4806a25df1a18c9ab709"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:04.803687 10368 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_284132ac952a4806a25df1a18c9ab709 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:04.806579 11040 tablet_bootstrap.cc:492] T 7463fcb18e884b1dbc478ebaa19f6cb4 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:04.806939 11040 ts_tablet_manager.cc:1397] T 7463fcb18e884b1dbc478ebaa19f6cb4 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.058s user 0.005s sys 0.002s
I20260924 00:21:04.807498 10365 catalog_manager.cc:5495] T 5b0345252c3c496db2987c30adce3da7 P b10a9082a37b46f09d581944914c1670 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8626e300d0644b0908797ad61bd7cef (127.10.4.67). New cstate: current_term: 1 leader_uuid: "c8626e300d0644b0908797ad61bd7cef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } }
I20260924 00:21:04.808599 10365 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 5b0345252c3c496db2987c30adce3da7 (table default.Table_a5d798f7819e403a96f27e98cb0af2bb [id=30a3f7c8fb9c46cf8b883eedae25d938]). Expected version 4 got 0
I20260924 00:21:04.810954 11040 raft_consensus.cc:348] T 7463fcb18e884b1dbc478ebaa19f6cb4 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:04.811655 11040 raft_consensus.cc:374] T 7463fcb18e884b1dbc478ebaa19f6cb4 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:04.811888 11040 raft_consensus.cc:729] T 7463fcb18e884b1dbc478ebaa19f6cb4 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:04.815423 10915 tablet_service.cc:1514] Processing DeleteTablet for tablet 675b9e33edcf4d4ea7fa428b476e3945 with delete_type TABLET_DATA_DELETED (replaced by tablet 915e8072851c4d2b90f8af12b1dc58e0) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:04.815754 10777 tablet_service.cc:1467] Processing CreateTablet for tablet 0de205104d7c4caf8f8f228e6a0d8c3a (DEFAULT_TABLE table=default.Table_6496e9c31f6349e78e6f469660e561d6 [id=237b25f430464714972a483a335f0a0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:04.816738 10777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0de205104d7c4caf8f8f228e6a0d8c3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:04.818634 11040 consensus_queue.cc:260] T 7463fcb18e884b1dbc478ebaa19f6cb4 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:04.820470 10773 tablet_service.cc:1467] Processing CreateTablet for tablet 8e628abffd9545b391c25f83c9903732 (DEFAULT_TABLE table=default.Table_dc51cf031b5e41ccb485d5db42ec823a [id=a95ec39c5f114ca79e47470b8a113881]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:04.821492 10773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e628abffd9545b391c25f83c9903732. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:04.749540 11051 raft_consensus.cc:2229] T 59c077eb68794340ac900dbedc7056ee P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:21:04.826987 11335 ts_tablet_manager.cc:1918] T 666755ec87bb496db75e43ec9204ec5b P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:04.827422 11335 log.cc:1196] T 666755ec87bb496db75e43ec9204ec5b P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/666755ec87bb496db75e43ec9204ec5b
I20260924 00:21:04.828135 11335 ts_tablet_manager.cc:1939] T 666755ec87bb496db75e43ec9204ec5b P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:04.817678 10625 tablet_service.cc:1467] Processing CreateTablet for tablet 0de205104d7c4caf8f8f228e6a0d8c3a (DEFAULT_TABLE table=default.Table_6496e9c31f6349e78e6f469660e561d6 [id=237b25f430464714972a483a335f0a0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:04.829159 11049 log.cc:1196] T 59c077eb68794340ac900dbedc7056ee P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/59c077eb68794340ac900dbedc7056ee
I20260924 00:21:04.829804 10915 tablet_service.cc:1514] Processing DeleteTablet for tablet a0f71049f6b3478da3b07ebc9293ce64 with delete_type TABLET_DATA_DELETED (replaced by tablet d574642ebc6340fcb5e83c36804ee9f8) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:04.830616 11049 ts_tablet_manager.cc:1939] T 59c077eb68794340ac900dbedc7056ee P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:04.831797 11049 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 146) took 1561 ms. Trace:
I20260924 00:21:04.832572 10915 tablet_service.cc:1514] Processing DeleteTablet for tablet d4e120acff3e4c30aed813b0b24fbfee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:03 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:04.832233 11049 rpcz_store.cc:276] 0924 00:21:03.270501 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:03.270612 (+ 111us) service_pool.cc:227] Handling call
0924 00:21:04.674697 (+1404085us) ts_tablet_manager.cc:1145] Deleting tablet 59c077eb68794340ac900dbedc7056ee
0924 00:21:04.674733 (+ 36us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:04.757351 (+ 82618us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:04.786560 (+ 29209us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:04.831716 (+ 45156us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1403553}
I20260924 00:21:04.834043 10906 tablet_service.cc:1514] Processing DeleteTablet for tablet 9319f9caadb04067a2e9700b5019ed37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:04 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:04.835345 11322 raft_consensus.cc:482] T 2964cd68d5e94f3780a6a3157bd86983 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:04.835723 11322 raft_consensus.cc:504] T 2964cd68d5e94f3780a6a3157bd86983 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:04.838102 11322 leader_election.cc:288] T 2964cd68d5e94f3780a6a3157bd86983 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:04.838888 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 59c077eb68794340ac900dbedc7056ee (table default.Table_5cf772cb831f4e7e97430617b962db80 [id=226555984f9741d3955c34512885c9cc]) successfully deleted
I20260924 00:21:04.841998 10371 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_08f63e23539246519606297527c945af"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:04.844040 10371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_08f63e23539246519606297527c945af in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:04.844285 10909 tablet_service.cc:1467] Processing CreateTablet for tablet d32a0c74b527451fb723e3851d5d055b (DEFAULT_TABLE table=default.Table_0dc486ba6e6d4091912ecb311b5dbd33 [id=a3f51096ae8b4b3188795babd04bd547]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:04.845374 10909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d32a0c74b527451fb723e3851d5d055b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:04.838282 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2964cd68d5e94f3780a6a3157bd86983" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:04.840235 10779 tablet_service.cc:1514] Processing DeleteTablet for tablet 7463fcb18e884b1dbc478ebaa19f6cb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:02 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:04.842303 11040 ts_tablet_manager.cc:1428] T 7463fcb18e884b1dbc478ebaa19f6cb4 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.035s user 0.001s sys 0.005s
W20260924 00:21:04.847421 10592 leader_election.cc:341] T 2964cd68d5e94f3780a6a3157bd86983 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:04.850631 10878 leader_election.cc:302] T 666755ec87bb496db75e43ec9204ec5b P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters: b10a9082a37b46f09d581944914c1670
I20260924 00:21:04.851347 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 5d7d52be4f1f4fc29b8172b9fa878f76 (table default.Table_451536c1b7444188b1dcab7ce70d663f [id=fde3781ccd3748f2acc7ac5f3f5064bf]) successfully deleted
I20260924 00:21:04.852149 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 666755ec87bb496db75e43ec9204ec5b (table default.Table_1347839140d544c3b4aee063e2efe906 [id=00b8c1d674cc460bb7b2425c55037246]) successfully deleted
I20260924 00:21:04.836105 10915 tablet_service.cc:1467] Processing CreateTablet for tablet 0de205104d7c4caf8f8f228e6a0d8c3a (DEFAULT_TABLE table=default.Table_6496e9c31f6349e78e6f469660e561d6 [id=237b25f430464714972a483a335f0a0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:04.839322 10939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2964cd68d5e94f3780a6a3157bd86983" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:04.854048 10625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0de205104d7c4caf8f8f228e6a0d8c3a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:21:04.856436 10589 leader_election.cc:341] T 2964cd68d5e94f3780a6a3157bd86983 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:04.856827 10589 leader_election.cc:302] T 2964cd68d5e94f3780a6a3157bd86983 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:04.856865 10372 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_c41522845ef54f468c7686bc6b171fe6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:04.858242 10372 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c41522845ef54f468c7686bc6b171fe6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:04.859367 11322 raft_consensus.cc:2726] T 2964cd68d5e94f3780a6a3157bd86983 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:04.868464 10373 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 0de205104d7c4caf8f8f228e6a0d8c3a
I20260924 00:21:04.873380 10634 tablet_service.cc:1514] Processing DeleteTablet for tablet 0de205104d7c4caf8f8f228e6a0d8c3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:04 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:04.877497 11049 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 147) took 1605 ms. Trace:
I20260924 00:21:04.877892 11049 rpcz_store.cc:276] 0924 00:21:03.271574 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:03.271678 (+ 104us) service_pool.cc:227] Handling call
0924 00:21:04.877128 (+1605450us) ts_tablet_manager.cc:1145] Deleting tablet 8a9443159a334093b4135364057e0c8b
0924 00:21:04.877165 (+ 37us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:04.877389 (+ 224us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1604940}
W20260924 00:21:04.879338 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 8a9443159a334093b4135364057e0c8b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8a9443159a334093b4135364057e0c8b already in progress: creating tablet
I20260924 00:21:04.879479 11342 raft_consensus.cc:2781] T 666755ec87bb496db75e43ec9204ec5b P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:04.879479 10634 tablet_service.cc:1467] Processing CreateTablet for tablet 8e628abffd9545b391c25f83c9903732 (DEFAULT_TABLE table=default.Table_dc51cf031b5e41ccb485d5db42ec823a [id=a95ec39c5f114ca79e47470b8a113881]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:04.879897 11342 raft_consensus.cc:482] T 666755ec87bb496db75e43ec9204ec5b P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:04.880168 11342 raft_consensus.cc:3037] T 666755ec87bb496db75e43ec9204ec5b P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:04.880374 10634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e628abffd9545b391c25f83c9903732. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:04.885140 10629 tablet_service.cc:1467] Processing CreateTablet for tablet d574642ebc6340fcb5e83c36804ee9f8 (DEFAULT_TABLE table=default.Table_9f2df24361da4f9a9340f6f8673043ea [id=59ec3e94e2f44150994a5ffc1d9fa8f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:04.886096 10629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d574642ebc6340fcb5e83c36804ee9f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:04.889050 10915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0de205104d7c4caf8f8f228e6a0d8c3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:04.898722 11384 tablet.cc:1722] T 85a0ad61b7cb4e11b30e84ddc4655b64 P c8626e300d0644b0908797ad61bd7cef: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260924 00:21:04.864890 11040 tablet_bootstrap.cc:492] T 0980afc2af174da2a8ca2dd414e9825c P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:04.905133 10626 tablet_service.cc:1467] Processing CreateTablet for tablet b1c2eb7623f847a3a5b6964627ab94df (DEFAULT_TABLE table=default.Table_aed2460280354a9b94c52d37815f3347 [id=32818e416d4a4677958734f2e7cc2047]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:04.906262 10626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1c2eb7623f847a3a5b6964627ab94df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:04.871244 10779 tablet_service.cc:1514] Processing DeleteTablet for tablet 0de205104d7c4caf8f8f228e6a0d8c3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:04 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:04.899176 10772 tablet_service.cc:1467] Processing CreateTablet for tablet d574642ebc6340fcb5e83c36804ee9f8 (DEFAULT_TABLE table=default.Table_9f2df24361da4f9a9340f6f8673043ea [id=59ec3e94e2f44150994a5ffc1d9fa8f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:04.902004 10774 tablet_service.cc:1467] Processing CreateTablet for tablet b1c2eb7623f847a3a5b6964627ab94df (DEFAULT_TABLE table=default.Table_aed2460280354a9b94c52d37815f3347 [id=32818e416d4a4677958734f2e7cc2047]), partition=RANGE (key) PARTITION UNBOUNDED
W20260924 00:21:04.909665 10878 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:35359, user_credentials={real_user=slave}} blocked reactor thread for 59333.7us
I20260924 00:21:04.912597 11040 tablet_bootstrap.cc:654] T 0980afc2af174da2a8ca2dd414e9825c P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:04.914135 11396 tablet.cc:1717] T 96960f3061fb43e9939b470f49edf1e9 P c8626e300d0644b0908797ad61bd7cef: Skipping requested alter to schema version 2, tablet already version 2
I20260924 00:21:04.853621 11335 tablet_replica.cc:330] T 7463fcb18e884b1dbc478ebaa19f6cb4 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:04.916285 11335 raft_consensus.cc:2229] T 7463fcb18e884b1dbc478ebaa19f6cb4 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:04.916812 11335 raft_consensus.cc:2258] T 7463fcb18e884b1dbc478ebaa19f6cb4 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:04.917724 11049 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 155) took 1134 ms. Trace:
I20260924 00:21:04.918095 11049 rpcz_store.cc:276] 0924 00:21:03.783506 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:03.783605 (+ 99us) service_pool.cc:227] Handling call
0924 00:21:04.917387 (+1133782us) ts_tablet_manager.cc:1145] Deleting tablet cd30f52583124543b5d616ef94b31831
0924 00:21:04.917432 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:04.917667 (+ 235us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1094728}
W20260924 00:21:04.910890 10878 leader_election.cc:341] T 195b033091b543b58ee08417eae2c0e6 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
W20260924 00:21:04.919389 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet cd30f52583124543b5d616ef94b31831 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cd30f52583124543b5d616ef94b31831 already in progress: creating tablet
I20260924 00:21:04.921036 11256 consensus_queue.cc:1035] T 5b0345252c3c496db2987c30adce3da7 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Connected to new peer: Peer: permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:04.927211 11049 tablet_replica.cc:330] T ffa011cf8dc446de849bea75a4f20056 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:04.927935 11049 raft_consensus.cc:2229] T ffa011cf8dc446de849bea75a4f20056 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:04.928316 11049 raft_consensus.cc:2258] T ffa011cf8dc446de849bea75a4f20056 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:04.929816 11049 ts_tablet_manager.cc:1905] T ffa011cf8dc446de849bea75a4f20056 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:04.931951 10370 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 5107b763acf04df88553ba0af2757570
I20260924 00:21:04.933164 10370 catalog_manager.cc:6565] ReplaceTablet: tablet 5107b763acf04df88553ba0af2757570 deleted and replaced by tablet cf0f0f016d074b1fb78d9b427c67fe86
W20260924 00:21:04.926362 10878 consensus_peers.cc:487] T fcf651adb1d3459facfe3a8df2a91ed7 P c8626e300d0644b0908797ad61bd7cef -> Peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Couldn't send request to peer 48fbd603abe146c585d15196e1b5be08. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260924 00:21:04.934523 10878 leader_election.cc:341] T 195b033091b543b58ee08417eae2c0e6 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:04.934580 10779 tablet_service.cc:1514] Processing DeleteTablet for tablet 5107b763acf04df88553ba0af2757570 with delete_type TABLET_DATA_DELETED (replaced by tablet cf0f0f016d074b1fb78d9b427c67fe86) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:04.934940 11335 ts_tablet_manager.cc:1905] T 7463fcb18e884b1dbc478ebaa19f6cb4 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260924 00:21:04.935206 10878 leader_election.cc:341] T f77aea0aad5a430f941aaec0a1c19640 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:04.941857 10632 tablet_service.cc:1514] Processing DeleteTablet for tablet 5107b763acf04df88553ba0af2757570 with delete_type TABLET_DATA_DELETED (replaced by tablet cf0f0f016d074b1fb78d9b427c67fe86) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:04.942444 11040 tablet_bootstrap.cc:492] T 0980afc2af174da2a8ca2dd414e9825c P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:04.949213 11040 ts_tablet_manager.cc:1397] T 0980afc2af174da2a8ca2dd414e9825c P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.084s user 0.008s sys 0.002s
I20260924 00:21:04.950073 10370 master_service.cc:946] ReplaceTablet: received request to replace tablet c7cb7c6892404c7fa593cdaaac67f800 from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:04.953039 11041 tablet_bootstrap.cc:492] T f3432ac96da94d5099bcf9764c2b32e1 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:04.953416 11041 ts_tablet_manager.cc:1397] T f3432ac96da94d5099bcf9764c2b32e1 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.320s user 0.026s sys 0.032s
I20260924 00:21:04.950675 11342 raft_consensus.cc:504] T 666755ec87bb496db75e43ec9204ec5b P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } }
I20260924 00:21:04.973001 10779 tablet_service.cc:1467] Processing CreateTablet for tablet 84e73180fab74524b4d568a010dc7175 (DEFAULT_TABLE table=default.Table_82ad1ad5194e43008bb85e5b99ae0d44 [id=8ae806d7df6c4158959efbe958fd4bcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:04.976714 11342 leader_election.cc:288] T 666755ec87bb496db75e43ec9204ec5b P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:21:04.964541 11039 tablet_bootstrap.cc:492] T 772fb3d6180e499daf94ceeb9b26048d P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:04.977552 11039 ts_tablet_manager.cc:1397] T 772fb3d6180e499daf94ceeb9b26048d P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.453s user 0.005s sys 0.007s
I20260924 00:21:04.978317 11040 raft_consensus.cc:348] T 0980afc2af174da2a8ca2dd414e9825c P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:04.978968 11040 raft_consensus.cc:374] T 0980afc2af174da2a8ca2dd414e9825c P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:04.979216 11040 raft_consensus.cc:729] T 0980afc2af174da2a8ca2dd414e9825c P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:04.979890 11039 raft_consensus.cc:348] T 772fb3d6180e499daf94ceeb9b26048d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:04.975060 11051 raft_consensus.cc:2258] T 59c077eb68794340ac900dbedc7056ee P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 00:21:04.922665 10876 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:33461, user_credentials={real_user=slave}} blocked reactor thread for 161700us
I20260924 00:21:04.981989 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f77aea0aad5a430f941aaec0a1c19640" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:04.982571 10654 raft_consensus.cc:2454] T f77aea0aad5a430f941aaec0a1c19640 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
I20260924 00:21:04.983067 10772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d574642ebc6340fcb5e83c36804ee9f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:04.985519 10373 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_b1726c09065e4fbaa2e6799b3ebdbc74" } new_table_name: "default.Table_66a5eb5613d743c1b96af8b5a252fd85" modify_external_catalogs: true
I20260924 00:21:04.987838 10910 tablet_service.cc:1514] Processing DeleteTablet for tablet 0de205104d7c4caf8f8f228e6a0d8c3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:04 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:04.989384 10369 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_ab9660d8c1a5491c8fee2f43483105a7" } new_table_name: "default.Table_8a934fde5e054e3eab58cfb899302830" modify_external_catalogs: true
I20260924 00:21:04.989183 11040 consensus_queue.cc:260] T 0980afc2af174da2a8ca2dd414e9825c P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:04.989446 10910 tablet_service.cc:1467] Processing CreateTablet for tablet 8e628abffd9545b391c25f83c9903732 (DEFAULT_TABLE table=default.Table_dc51cf031b5e41ccb485d5db42ec823a [id=a95ec39c5f114ca79e47470b8a113881]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:04.991063 10910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e628abffd9545b391c25f83c9903732. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:04.992977 10366 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_d853fb40e0554242976678966283153e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:04.993726 11040 ts_tablet_manager.cc:1428] T 0980afc2af174da2a8ca2dd414e9825c P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.044s user 0.005s sys 0.000s
W20260924 00:21:04.994382 10366 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d853fb40e0554242976678966283153e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:04.998564 10779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84e73180fab74524b4d568a010dc7175. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:04.998446 11041 raft_consensus.cc:348] T f3432ac96da94d5099bcf9764c2b32e1 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:04.999073 11041 raft_consensus.cc:374] T f3432ac96da94d5099bcf9764c2b32e1 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:04.999320 11041 raft_consensus.cc:729] T f3432ac96da94d5099bcf9764c2b32e1 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:04.999945 11041 consensus_queue.cc:260] T f3432ac96da94d5099bcf9764c2b32e1 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:05.005252 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1c2eb7623f847a3a5b6964627ab94df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:05.006873 10876 leader_election.cc:302] T f77aea0aad5a430f941aaec0a1c19640 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08
I20260924 00:21:05.008383 11341 raft_consensus.cc:2781] T f77aea0aad5a430f941aaec0a1c19640 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:05.008682 11341 raft_consensus.cc:482] T f77aea0aad5a430f941aaec0a1c19640 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:05.008970 11341 raft_consensus.cc:3037] T f77aea0aad5a430f941aaec0a1c19640 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
W20260924 00:21:05.025396 11403 log.cc:924] Time spent T 59c077eb68794340ac900dbedc7056ee P c8626e300d0644b0908797ad61bd7cef: Append to log took a long time: real 0.053s user 0.000s sys 0.000s
I20260924 00:21:05.027413 11051 ts_tablet_manager.cc:1905] T 59c077eb68794340ac900dbedc7056ee P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:05.042196 10908 tablet_service.cc:1467] Processing CreateTablet for tablet d574642ebc6340fcb5e83c36804ee9f8 (DEFAULT_TABLE table=default.Table_9f2df24361da4f9a9340f6f8673043ea [id=59ec3e94e2f44150994a5ffc1d9fa8f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:05.043679 10908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d574642ebc6340fcb5e83c36804ee9f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:05.045943 11049 ts_tablet_manager.cc:1918] T ffa011cf8dc446de849bea75a4f20056 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:05.046298 11049 log.cc:1196] T ffa011cf8dc446de849bea75a4f20056 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/ffa011cf8dc446de849bea75a4f20056
I20260924 00:21:05.046865 11049 ts_tablet_manager.cc:1939] T ffa011cf8dc446de849bea75a4f20056 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:05.050444 10367 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 85a0ad61b7cb4e11b30e84ddc4655b64 (table default.Table_67493ad72aac4bbca26a6a322a60e5cd [id=12285914811f4220822b47a2a62b01ab]). Expected version 2 got 0
I20260924 00:21:05.051095 10367 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 96960f3061fb43e9939b470f49edf1e9 (table default.Table_477c0f1cce9a4c089551e4e767ba7ae0 [id=6251651eb5944ee086dd564d86204148]). Expected version 2 got 0
I20260924 00:21:05.051644 10367 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 5b0345252c3c496db2987c30adce3da7 (table default.Table_a5d798f7819e403a96f27e98cb0af2bb [id=30a3f7c8fb9c46cf8b883eedae25d938]). Expected version 4 got 0
I20260924 00:21:05.052302 10367 catalog_manager.cc:5495] T fcf651adb1d3459facfe3a8df2a91ed7 P c8626e300d0644b0908797ad61bd7cef reported cstate change: term changed from 0 to 1, leader changed from <none> to c8626e300d0644b0908797ad61bd7cef (127.10.4.67). New cstate: current_term: 1 leader_uuid: "c8626e300d0644b0908797ad61bd7cef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } health_report { overall_health: UNKNOWN } } }
I20260924 00:21:05.061834 10907 tablet_service.cc:1514] Processing DeleteTablet for tablet 5107b763acf04df88553ba0af2757570 with delete_type TABLET_DATA_DELETED (replaced by tablet cf0f0f016d074b1fb78d9b427c67fe86) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:05.062896 10625 tablet_service.cc:1467] Processing CreateTablet for tablet 84e73180fab74524b4d568a010dc7175 (DEFAULT_TABLE table=default.Table_82ad1ad5194e43008bb85e5b99ae0d44 [id=8ae806d7df6c4158959efbe958fd4bcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:05.063405 10907 tablet_service.cc:1467] Processing CreateTablet for tablet 84e73180fab74524b4d568a010dc7175 (DEFAULT_TABLE table=default.Table_82ad1ad5194e43008bb85e5b99ae0d44 [id=8ae806d7df6c4158959efbe958fd4bcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:05.063951 10625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84e73180fab74524b4d568a010dc7175. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:05.064177 11040 tablet_bootstrap.cc:492] T 6ac9c7da7fb546f6ae935371799cbad6 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:05.064373 10907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84e73180fab74524b4d568a010dc7175. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:05.067544 10634 tablet_service.cc:1467] Processing CreateTablet for tablet 0c5e5e375e414cd8b268e33b0c13b830 (DEFAULT_TABLE table=default.Table_f19e1bc3467b4ac9943c4873aba523de [id=f45aef19d67844c3b266bc5ae753c480]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:05.068444 10634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c5e5e375e414cd8b268e33b0c13b830. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:05.071666 11040 tablet_bootstrap.cc:654] T 6ac9c7da7fb546f6ae935371799cbad6 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:05.076289 11039 raft_consensus.cc:374] T 772fb3d6180e499daf94ceeb9b26048d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:05.076625 11039 raft_consensus.cc:729] T 772fb3d6180e499daf94ceeb9b26048d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:05.076897 11335 ts_tablet_manager.cc:1918] T 7463fcb18e884b1dbc478ebaa19f6cb4 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:05.077419 11335 log.cc:1196] T 7463fcb18e884b1dbc478ebaa19f6cb4 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/7463fcb18e884b1dbc478ebaa19f6cb4
I20260924 00:21:05.078092 11335 ts_tablet_manager.cc:1939] T 7463fcb18e884b1dbc478ebaa19f6cb4 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:05.079932 11341 raft_consensus.cc:504] T f77aea0aad5a430f941aaec0a1c19640 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
W20260924 00:21:05.081040 10876 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:33461, user_credentials={real_user=slave}} blocked reactor thread for 72865.7us
W20260924 00:21:05.081629 10876 leader_election.cc:341] T 666755ec87bb496db75e43ec9204ec5b P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: 666755ec87bb496db75e43ec9204ec5b
I20260924 00:21:05.082101 10773 tablet_service.cc:1467] Processing CreateTablet for tablet 0c5e5e375e414cd8b268e33b0c13b830 (DEFAULT_TABLE table=default.Table_f19e1bc3467b4ac9943c4873aba523de [id=f45aef19d67844c3b266bc5ae753c480]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:05.089533 10773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c5e5e375e414cd8b268e33b0c13b830. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:05.093103 10906 tablet_service.cc:1467] Processing CreateTablet for tablet 0c5e5e375e414cd8b268e33b0c13b830 (DEFAULT_TABLE table=default.Table_f19e1bc3467b4ac9943c4873aba523de [id=f45aef19d67844c3b266bc5ae753c480]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:05.094128 10906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c5e5e375e414cd8b268e33b0c13b830. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:05.092844 10799 raft_consensus.cc:3037] T f294831873d548e4b9201c1bbb964096 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:05.099339 11040 tablet_bootstrap.cc:492] T 6ac9c7da7fb546f6ae935371799cbad6 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:05.099701 11040 ts_tablet_manager.cc:1397] T 6ac9c7da7fb546f6ae935371799cbad6 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.036s user 0.009s sys 0.001s
I20260924 00:21:05.102283 11040 raft_consensus.cc:348] T 6ac9c7da7fb546f6ae935371799cbad6 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:05.102901 11040 raft_consensus.cc:374] T 6ac9c7da7fb546f6ae935371799cbad6 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:05.103127 11040 raft_consensus.cc:729] T 6ac9c7da7fb546f6ae935371799cbad6 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:05.103653 11040 consensus_queue.cc:260] T 6ac9c7da7fb546f6ae935371799cbad6 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:05.107012 11039 consensus_queue.cc:260] T 772fb3d6180e499daf94ceeb9b26048d P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:05.108835 11039 ts_tablet_manager.cc:1428] T 772fb3d6180e499daf94ceeb9b26048d P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.131s user 0.004s sys 0.000s
I20260924 00:21:05.115386 11040 ts_tablet_manager.cc:1428] T 6ac9c7da7fb546f6ae935371799cbad6 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.015s user 0.005s sys 0.001s
I20260924 00:21:05.133420 11040 tablet_bootstrap.cc:492] T f77aea0aad5a430f941aaec0a1c19640 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:05.136412 11049 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 158) took 1283 ms. Trace:
I20260924 00:21:05.136760 11049 rpcz_store.cc:276] 0924 00:21:03.852916 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:03.853068 (+ 152us) service_pool.cc:227] Handling call
0924 00:21:04.925408 (+1072340us) ts_tablet_manager.cc:1145] Deleting tablet ffa011cf8dc446de849bea75a4f20056
0924 00:21:04.925494 (+ 86us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:05.017568 (+ 92074us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:05.045739 (+ 28171us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:05.136346 (+ 90607us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":87673,"tablet-delete.queue_time_us":1071532}
I20260924 00:21:05.138777 11049 tablet_replica.cc:330] T b06bf1e4691441769e162aa5404d4656 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:05.139426 11049 raft_consensus.cc:2229] T b06bf1e4691441769e162aa5404d4656 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:05.139835 11049 raft_consensus.cc:2258] T b06bf1e4691441769e162aa5404d4656 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:05.141644 11049 ts_tablet_manager.cc:1905] T b06bf1e4691441769e162aa5404d4656 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:05.144214 11041 ts_tablet_manager.cc:1428] T f3432ac96da94d5099bcf9764c2b32e1 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.191s user 0.005s sys 0.001s
I20260924 00:21:05.146843 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet ffa011cf8dc446de849bea75a4f20056 (table default.Table_6cf69012192f4f1087e29db70ae8c282 [id=93c1734de58245f8b1d5ab88ec9f0c8e]) successfully deleted
W20260924 00:21:05.147954 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet d4e120acff3e4c30aed813b0b24fbfee because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d4e120acff3e4c30aed813b0b24fbfee already in progress: creating tablet
I20260924 00:21:05.150543 10626 tablet_service.cc:1467] Processing CreateTablet for tablet 915e8072851c4d2b90f8af12b1dc58e0 (DEFAULT_TABLE table=default.Table_ee73693b7a6e46a3af53f7828da00f9e [id=9359b05533064c38b9bd589c0bb06471]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:05.151598 10626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 915e8072851c4d2b90f8af12b1dc58e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:05.156235 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 7463fcb18e884b1dbc478ebaa19f6cb4 (table default.Table_d01ccf505bd94805b62c8f0755dad1d1 [id=8e2fe68fe5da4414b7269f526b8af683]) successfully deleted
W20260924 00:21:05.157310 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 0de205104d7c4caf8f8f228e6a0d8c3a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0de205104d7c4caf8f8f228e6a0d8c3a already in progress: creating tablet
W20260924 00:21:05.158226 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 5107b763acf04df88553ba0af2757570 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5107b763acf04df88553ba0af2757570 already in progress: creating tablet
I20260924 00:21:05.167132 10368 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_0cdc660142f44fb5b77963f9432b4182" } modify_external_catalogs: true
I20260924 00:21:05.168111 10368 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_0cdc660142f44fb5b77963f9432b4182" } modify_external_catalogs: true
W20260924 00:21:05.179210 10878 leader_election.cc:341] T 666755ec87bb496db75e43ec9204ec5b P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Not found: Tablet not found: 666755ec87bb496db75e43ec9204ec5b
I20260924 00:21:05.179585 10878 leader_election.cc:302] T 666755ec87bb496db75e43ec9204ec5b P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670
I20260924 00:21:05.180202 11342 raft_consensus.cc:2726] T 666755ec87bb496db75e43ec9204ec5b P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260924 00:21:05.201011 10374 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet d32a0c74b527451fb723e3851d5d055b
I20260924 00:21:05.205058 11039 tablet_bootstrap.cc:492] T 6ac9c7da7fb546f6ae935371799cbad6 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:05.205334 10629 tablet_service.cc:1514] Processing DeleteTablet for tablet d32a0c74b527451fb723e3851d5d055b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:04 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:05.208030 11051 ts_tablet_manager.cc:1918] T 59c077eb68794340ac900dbedc7056ee P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:05.208518 11051 log.cc:1196] T 59c077eb68794340ac900dbedc7056ee P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/59c077eb68794340ac900dbedc7056ee
I20260924 00:21:05.209288 11051 ts_tablet_manager.cc:1939] T 59c077eb68794340ac900dbedc7056ee P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:05.227568 10372 master_service.cc:946] ReplaceTablet: received request to replace tablet b36cbb6520de4df58b88cfa53c20ee7a from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:05.229573 10912 tablet_service.cc:1514] Processing DeleteTablet for tablet d32a0c74b527451fb723e3851d5d055b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:04 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:05.233819 11341 leader_election.cc:288] T f77aea0aad5a430f941aaec0a1c19640 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:05.236402 10651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f77aea0aad5a430f941aaec0a1c19640" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:21:05.236989 10651 raft_consensus.cc:3037] T f77aea0aad5a430f941aaec0a1c19640 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:05.244035 10370 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet c7cb7c6892404c7fa593cdaaac67f800
W20260924 00:21:05.244647 10369 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a62b85ebb2b84ae19cad3e3e2dd07a59 consensus state has no leader
I20260924 00:21:05.245534 10370 catalog_manager.cc:6565] ReplaceTablet: tablet c7cb7c6892404c7fa593cdaaac67f800 deleted and replaced by tablet df03f899e5de4c12b16fbb40bdb1f33a
I20260924 00:21:05.250105 11040 tablet_bootstrap.cc:654] T f77aea0aad5a430f941aaec0a1c19640 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:05.250830 10651 raft_consensus.cc:2454] T f77aea0aad5a430f941aaec0a1c19640 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 1.
I20260924 00:21:05.252115 10876 leader_election.cc:302] T f77aea0aad5a430f941aaec0a1c19640 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:05.254693 10629 tablet_service.cc:1514] Processing DeleteTablet for tablet c7cb7c6892404c7fa593cdaaac67f800 with delete_type TABLET_DATA_DELETED (replaced by tablet df03f899e5de4c12b16fbb40bdb1f33a) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:05.262246 11323 raft_consensus.cc:2781] T f77aea0aad5a430f941aaec0a1c19640 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:05.262676 11323 raft_consensus.cc:686] T f77aea0aad5a430f941aaec0a1c19640 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Becoming Leader. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Running, Role: LEADER
I20260924 00:21:05.263388 11323 consensus_queue.cc:237] T f77aea0aad5a430f941aaec0a1c19640 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:05.263913 10797 raft_consensus.cc:3037] T fcf651adb1d3459facfe3a8df2a91ed7 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:05.265164 10772 tablet_service.cc:1514] Processing DeleteTablet for tablet d32a0c74b527451fb723e3851d5d055b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:04 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:05.265431 10798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f77aea0aad5a430f941aaec0a1c19640" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:21:05.265662 10779 tablet_service.cc:1467] Processing CreateTablet for tablet 915e8072851c4d2b90f8af12b1dc58e0 (DEFAULT_TABLE table=default.Table_ee73693b7a6e46a3af53f7828da00f9e [id=9359b05533064c38b9bd589c0bb06471]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:05.265164 10777 tablet_service.cc:1514] Processing DeleteTablet for tablet c7cb7c6892404c7fa593cdaaac67f800 with delete_type TABLET_DATA_DELETED (replaced by tablet df03f899e5de4c12b16fbb40bdb1f33a) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:05.266645 10779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 915e8072851c4d2b90f8af12b1dc58e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:05.268343 11051 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35358 (request call id 145) took 1780 ms. Trace:
I20260924 00:21:05.268628 10774 tablet_service.cc:1514] Processing DeleteTablet for tablet 7463fcb18e884b1dbc478ebaa19f6cb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:02 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:05.268647 11051 rpcz_store.cc:276] 0924 00:21:03.487705 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:03.877137 (+389432us) service_pool.cc:227] Handling call
0924 00:21:04.748670 (+871533us) ts_tablet_manager.cc:1145] Deleting tablet 59c077eb68794340ac900dbedc7056ee
0924 00:21:04.748709 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:05.034155 (+285446us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:05.170495 (+136340us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:05.268285 (+ 97790us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":870541}
I20260924 00:21:05.269948 11051 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35358 (request call id 146) took 1403 ms. Trace:
I20260924 00:21:05.270305 11051 rpcz_store.cc:276] 0924 00:21:03.866484 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:03.882008 (+ 15524us) service_pool.cc:227] Handling call
0924 00:21:05.269625 (+1387617us) ts_tablet_manager.cc:1145] Deleting tablet 8a9443159a334093b4135364057e0c8b
0924 00:21:05.269666 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:05.269898 (+ 232us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1387035}
I20260924 00:21:05.271440 11051 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35358 (request call id 154) took 1191 ms. Trace:
I20260924 00:21:05.274951 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 59c077eb68794340ac900dbedc7056ee (table default.Table_5cf772cb831f4e7e97430617b962db80 [id=226555984f9741d3955c34512885c9cc]) successfully deleted
W20260924 00:21:05.275817 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 8a9443159a334093b4135364057e0c8b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8a9443159a334093b4135364057e0c8b already in progress: creating tablet
W20260924 00:21:05.282181 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet d32a0c74b527451fb723e3851d5d055b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d32a0c74b527451fb723e3851d5d055b already in progress: creating tablet
W20260924 00:21:05.283983 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet c7cb7c6892404c7fa593cdaaac67f800 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c7cb7c6892404c7fa593cdaaac67f800 already in progress: creating tablet
W20260924 00:21:05.285933 10361 catalog_manager.cc:4808] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 7463fcb18e884b1dbc478ebaa19f6cb4 because the tablet was not found. No further retry: Not found: Tablet not found: 7463fcb18e884b1dbc478ebaa19f6cb4
I20260924 00:21:05.272444 10908 tablet_service.cc:1467] Processing CreateTablet for tablet 915e8072851c4d2b90f8af12b1dc58e0 (DEFAULT_TABLE table=default.Table_ee73693b7a6e46a3af53f7828da00f9e [id=9359b05533064c38b9bd589c0bb06471]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:05.289387 10908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 915e8072851c4d2b90f8af12b1dc58e0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:21:05.274847 10878 leader_election.cc:341] T f77aea0aad5a430f941aaec0a1c19640 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:05.263028 11039 tablet_bootstrap.cc:654] T 6ac9c7da7fb546f6ae935371799cbad6 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:05.286876 10910 tablet_service.cc:1514] Processing DeleteTablet for tablet c7cb7c6892404c7fa593cdaaac67f800 with delete_type TABLET_DATA_DELETED (replaced by tablet df03f899e5de4c12b16fbb40bdb1f33a) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:05.287230 11051 rpcz_store.cc:276] 0924 00:21:04.079735 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:04.081742 (+ 2007us) service_pool.cc:227] Handling call
0924 00:21:05.271144 (+1189402us) ts_tablet_manager.cc:1145] Deleting tablet cd30f52583124543b5d616ef94b31831
0924 00:21:05.271179 (+ 35us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:05.271395 (+ 216us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1119991}
W20260924 00:21:05.299289 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet cd30f52583124543b5d616ef94b31831 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cd30f52583124543b5d616ef94b31831 already in progress: creating tablet
W20260924 00:21:05.301118 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 717759eb61674c89b05b8d2b0dc34979 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 717759eb61674c89b05b8d2b0dc34979 already in progress: creating tablet
W20260924 00:21:05.301867 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 675b9e33edcf4d4ea7fa428b476e3945 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 675b9e33edcf4d4ea7fa428b476e3945 already in progress: creating tablet
W20260924 00:21:05.302587 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet a0f71049f6b3478da3b07ebc9293ce64 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a0f71049f6b3478da3b07ebc9293ce64 already in progress: creating tablet
W20260924 00:21:05.303202 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet d4e120acff3e4c30aed813b0b24fbfee because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d4e120acff3e4c30aed813b0b24fbfee already in progress: creating tablet
W20260924 00:21:05.303823 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 9319f9caadb04067a2e9700b5019ed37 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9319f9caadb04067a2e9700b5019ed37 already in progress: creating tablet
I20260924 00:21:05.319410 11040 tablet_bootstrap.cc:492] T f77aea0aad5a430f941aaec0a1c19640 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:05.319805 11040 ts_tablet_manager.cc:1397] T f77aea0aad5a430f941aaec0a1c19640 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.194s user 0.006s sys 0.007s
I20260924 00:21:05.321311 11039 tablet_bootstrap.cc:492] T 6ac9c7da7fb546f6ae935371799cbad6 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:05.321686 11039 ts_tablet_manager.cc:1397] T 6ac9c7da7fb546f6ae935371799cbad6 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.117s user 0.007s sys 0.002s
I20260924 00:21:05.323709 11039 raft_consensus.cc:348] T 6ac9c7da7fb546f6ae935371799cbad6 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:05.324330 11039 raft_consensus.cc:374] T 6ac9c7da7fb546f6ae935371799cbad6 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:05.324561 11039 raft_consensus.cc:729] T 6ac9c7da7fb546f6ae935371799cbad6 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:05.337216 11040 raft_consensus.cc:348] T f77aea0aad5a430f941aaec0a1c19640 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:05.337888 11040 raft_consensus.cc:374] T f77aea0aad5a430f941aaec0a1c19640 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:05.338042 11049 ts_tablet_manager.cc:1918] T b06bf1e4691441769e162aa5404d4656 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:05.338117 11040 raft_consensus.cc:729] T f77aea0aad5a430f941aaec0a1c19640 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:05.338387 11049 log.cc:1196] T b06bf1e4691441769e162aa5404d4656 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/b06bf1e4691441769e162aa5404d4656
I20260924 00:21:05.338979 11049 ts_tablet_manager.cc:1939] T b06bf1e4691441769e162aa5404d4656 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:05.338651 11040 consensus_queue.cc:260] T f77aea0aad5a430f941aaec0a1c19640 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:05.339509 11039 consensus_queue.cc:260] T 6ac9c7da7fb546f6ae935371799cbad6 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:05.345124 11323 raft_consensus.cc:482] T 6ac9c7da7fb546f6ae935371799cbad6 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:05.345534 11323 raft_consensus.cc:504] T 6ac9c7da7fb546f6ae935371799cbad6 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:05.347357 11323 leader_election.cc:288] T 6ac9c7da7fb546f6ae935371799cbad6 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:05.348140 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ac9c7da7fb546f6ae935371799cbad6" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:05.348732 10795 raft_consensus.cc:2454] T 6ac9c7da7fb546f6ae935371799cbad6 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
W20260924 00:21:05.349699 10876 leader_election.cc:341] T 6ac9c7da7fb546f6ae935371799cbad6 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: 6ac9c7da7fb546f6ae935371799cbad6
I20260924 00:21:05.348975 10371 catalog_manager.cc:5495] T 195b033091b543b58ee08417eae2c0e6 P c8626e300d0644b0908797ad61bd7cef reported cstate change: term changed from 0 to 1, leader changed from <none> to c8626e300d0644b0908797ad61bd7cef (127.10.4.67). New cstate: current_term: 1 leader_uuid: "c8626e300d0644b0908797ad61bd7cef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } health_report { overall_health: HEALTHY } } }
I20260924 00:21:05.350399 10371 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 195b033091b543b58ee08417eae2c0e6 (table default.Table_ca083375f0df4255b8654ed88040dfcd [id=f79ca8d11aa04d5f8f72d58b798ed4ea]). Expected version 1 got 0
I20260924 00:21:05.350507 10878 leader_election.cc:302] T 6ac9c7da7fb546f6ae935371799cbad6 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters: b10a9082a37b46f09d581944914c1670
I20260924 00:21:05.351342 10371 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 5b0345252c3c496db2987c30adce3da7 (table default.Table_a5d798f7819e403a96f27e98cb0af2bb [id=30a3f7c8fb9c46cf8b883eedae25d938]). Expected version 4 got 0
I20260924 00:21:05.351477 11323 raft_consensus.cc:2781] T 6ac9c7da7fb546f6ae935371799cbad6 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:05.351778 11323 raft_consensus.cc:482] T 6ac9c7da7fb546f6ae935371799cbad6 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:05.352061 11323 raft_consensus.cc:3037] T 6ac9c7da7fb546f6ae935371799cbad6 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:05.356552 10365 catalog_manager.cc:6040] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: default.Table_de36c9f702024914965bc6b371eda68e [id=e2cdad6dfe7846ebb3c14f5611cf0f18] alter complete (version 1)
W20260924 00:21:05.357877 10365 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.4.65:55561 (request call id 50) took 933 ms (client timeout 1000 ms). Trace:
I20260924 00:21:05.359229 11039 ts_tablet_manager.cc:1428] T 6ac9c7da7fb546f6ae935371799cbad6 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.037s user 0.005s sys 0.001s
I20260924 00:21:05.359781 11322 raft_consensus.cc:482] T f3432ac96da94d5099bcf9764c2b32e1 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:05.360193 11322 raft_consensus.cc:504] T f3432ac96da94d5099bcf9764c2b32e1 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:05.361310 11040 ts_tablet_manager.cc:1428] T f77aea0aad5a430f941aaec0a1c19640 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.038s user 0.005s sys 0.000s
I20260924 00:21:05.361994 11322 leader_election.cc:288] T f3432ac96da94d5099bcf9764c2b32e1 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:05.362186 11040 tablet_bootstrap.cc:492] T 772fb3d6180e499daf94ceeb9b26048d P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:05.364391 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3432ac96da94d5099bcf9764c2b32e1" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:05.365837 10592 leader_election.cc:341] T f3432ac96da94d5099bcf9764c2b32e1 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:05.365871 10373 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_001f95fae49849f88b710b424fe7714c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:05.366935 11323 raft_consensus.cc:504] T 6ac9c7da7fb546f6ae935371799cbad6 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
W20260924 00:21:05.367417 10373 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_001f95fae49849f88b710b424fe7714c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:05.367972 11071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
W20260924 00:21:05.358171 10365 rpcz_store.cc:269] 0924 00:21:04.424135 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:04.424487 (+ 352us) service_pool.cc:227] Handling call
0924 00:21:05.357817 (+933330us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 00:21:04.965605 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 00:21:04.965953 (+ 348us) write_op.cc:432] Acquiring schema lock in shared mode
0924 00:21:04.965976 (+ 23us) write_op.cc:435] Acquired schema lock
0924 00:21:04.965989 (+ 13us) tablet.cc:660] Decoding operations
0924 00:21:04.966220 (+ 231us) write_op.cc:620] Acquiring the partition lock for write op
0924 00:21:04.966258 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0924 00:21:04.966279 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0924 00:21:04.966408 (+ 129us) tablet.cc:699] Row locks acquired
0924 00:21:04.966420 (+ 12us) write_op.cc:260] PREPARE: finished
0924 00:21:04.966516 (+ 96us) write_op.cc:270] Start()
0924 00:21:04.966589 (+ 73us) write_op.cc:276] Timestamp: P: 1790209264966450 usec, L: 0
0924 00:21:04.966603 (+ 14us) op_driver.cc:331] REPLICATION: starting
0924 00:21:04.967040 (+ 437us) log.cc:842] Serialized 527 byte log entry
0924 00:21:05.040076 (+ 73036us) op_driver.cc:444] REPLICATION: finished
0924 00:21:05.045022 (+ 4946us) write_op.cc:301] APPLY: starting
0924 00:21:05.045079 (+ 57us) tablet.cc:1365] starting BulkCheckPresence
0924 00:21:05.045165 (+ 86us) tablet.cc:1368] finished BulkCheckPresence
0924 00:21:05.045252 (+ 87us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 00:21:05.045398 (+ 146us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 00:21:05.045519 (+ 121us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0924 00:21:05.045559 (+ 40us) write_op.cc:312] APPLY: finished
0924 00:21:05.045730 (+ 171us) log.cc:842] Serialized 23 byte log entry
0924 00:21:05.046318 (+ 588us) write_op.cc:489] Releasing partition, row and schema locks
0924 00:21:05.046380 (+ 62us) write_op.cc:454] Released schema lock
0924 00:21:05.046410 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0924 00:21:05.053321 (+ 6911us) write_op.cc:489] Releasing partition, row and schema locks
0924 00:21:05.053345 (+ 24us) write_op.cc:454] Released schema lock
Related trace 'op':
0924 00:21:05.286967 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 00:21:05.287423 (+ 456us) write_op.cc:432] Acquiring schema lock in shared mode
0924 00:21:05.287447 (+ 24us) write_op.cc:435] Acquired schema lock
0924 00:21:05.287460 (+ 13us) tablet.cc:660] Decoding operations
0924 00:21:05.287797 (+ 337us) write_op.cc:620] Acquiring the partition lock for write op
0924 00:21:05.287836 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0924 00:21:05.287858 (+ 22us) tablet.cc:683] Acquiring locks for 1 operations
0924 00:21:05.287996 (+ 138us) tablet.cc:699] Row locks acquired
0924 00:21:05.288009 (+ 13us) write_op.cc:260] PREPARE: finished
0924 00:21:05.288161 (+ 152us) write_op.cc:270] Start()
0924 00:21:05.288241 (+ 80us) write_op.cc:276] Timestamp: P: 1790209265288056 usec, L: 0
0924 00:21:05.288255 (+ 14us) op_driver.cc:331] REPLICATION: starting
0924 00:21:05.297584 (+ 9329us) log.cc:842] Serialized 356 byte log entry
0924 00:21:05.306990 (+ 9406us) op_driver.cc:444] REPLICATION: finished
0924 00:21:05.307965 (+ 975us) write_op.cc:301] APPLY: starting
0924 00:21:05.308017 (+ 52us) tablet.cc:1365] starting BulkCheckPresence
0924 00:21:05.308107 (+ 90us) tablet.cc:1368] finished BulkCheckPresence
0924 00:21:05.308199 (+ 92us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 00:21:05.308389 (+ 190us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 00:21:05.308512 (+ 123us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0924 00:21:05.308555 (+ 43us) write_op.cc:312] APPLY: finished
0924 00:21:05.308728 (+ 173us) log.cc:842] Serialized 23 byte log entry
0924 00:21:05.309298 (+ 570us) write_op.cc:489] Releasing partition, row and schema locks
0924 00:21:05.309354 (+ 56us) write_op.cc:454] Released schema lock
0924 00:21:05.309384 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0924 00:21:05.309682 (+ 298us) write_op.cc:489] Releasing partition, row and schema locks
0924 00:21:05.309705 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":4753,"apply.run_cpu_time_us":1376,"apply.run_wall_time_us":8520,"num_ops":1,"prepare.queue_time_us":155042,"prepare.run_cpu_time_us":1775,"prepare.run_wall_time_us":92068,"raft.queue_time_us":178,"raft.run_cpu_time_us":698,"raft.run_wall_time_us":699,"replication_time_us":73402}],["op",{"apply.queue_time_us":823,"apply.run_cpu_time_us":1341,"apply.run_wall_time_us":1885,"num_ops":1,"prepare.queue_time_us":238367,"prepare.run_cpu_time_us":2034,"prepare.run_wall_time_us":10866,"replication_time_us":18655}]]}
I20260924 00:21:05.372399 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3432ac96da94d5099bcf9764c2b32e1" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
W20260924 00:21:05.373764 10589 leader_election.cc:341] T f3432ac96da94d5099bcf9764c2b32e1 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:05.374126 10589 leader_election.cc:302] T f3432ac96da94d5099bcf9764c2b32e1 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:05.375185 11322 raft_consensus.cc:2726] T f3432ac96da94d5099bcf9764c2b32e1 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:05.378882 11039 tablet_bootstrap.cc:492] T 0980afc2af174da2a8ca2dd414e9825c P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:05.387586 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ac9c7da7fb546f6ae935371799cbad6" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
W20260924 00:21:05.304898 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 0de205104d7c4caf8f8f228e6a0d8c3a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0de205104d7c4caf8f8f228e6a0d8c3a already in progress: creating tablet
I20260924 00:21:05.388139 10795 raft_consensus.cc:3037] T 6ac9c7da7fb546f6ae935371799cbad6 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:05.388057 11342 consensus_queue.cc:1035] T 195b033091b543b58ee08417eae2c0e6 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Connected to new peer: Peer: permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260924 00:21:05.388420 10358 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.10.4.67:41301, user_credentials={real_user=slave}} blocked reactor thread for 83555.9us
W20260924 00:21:05.389106 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 5107b763acf04df88553ba0af2757570 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5107b763acf04df88553ba0af2757570 already in progress: creating tablet
W20260924 00:21:05.389643 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet d32a0c74b527451fb723e3851d5d055b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d32a0c74b527451fb723e3851d5d055b already in progress: creating tablet
W20260924 00:21:05.390280 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet c7cb7c6892404c7fa593cdaaac67f800 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c7cb7c6892404c7fa593cdaaac67f800 already in progress: creating tablet
I20260924 00:21:05.393181 11041 tablet_bootstrap.cc:492] T bf144ab1a6464a4ab02ac0642c5255e3 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:05.395815 11049 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 171) took 1083 ms. Trace:
I20260924 00:21:05.396144 11049 rpcz_store.cc:276] 0924 00:21:04.312048 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:04.316966 (+ 4918us) service_pool.cc:227] Handling call
0924 00:21:05.138636 (+821670us) ts_tablet_manager.cc:1145] Deleting tablet b06bf1e4691441769e162aa5404d4656
0924 00:21:05.138670 (+ 34us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:05.280713 (+142043us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:05.337830 (+ 57117us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:05.395757 (+ 57927us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":54966,"tablet-delete.queue_time_us":820278}
I20260924 00:21:05.397431 11049 tablet_replica.cc:330] T 0d555f059f84478eb805f4ca4534dab4 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:05.399034 10910 tablet_service.cc:1514] Processing DeleteTablet for tablet 666755ec87bb496db75e43ec9204ec5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:01 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:05.400027 10910 tablet_service.cc:1514] Processing DeleteTablet for tablet 59c077eb68794340ac900dbedc7056ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:02 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:05.401065 11051 tablet_replica.cc:330] T 666755ec87bb496db75e43ec9204ec5b P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:05.401777 11051 raft_consensus.cc:2229] T 666755ec87bb496db75e43ec9204ec5b P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:05.402338 11051 raft_consensus.cc:2258] T 666755ec87bb496db75e43ec9204ec5b P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:05.404539 11051 ts_tablet_manager.cc:1905] T 666755ec87bb496db75e43ec9204ec5b P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:05.407142 10366 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_2c873f940f8249f9950e78d583d64e36"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:05.408564 10366 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2c873f940f8249f9950e78d583d64e36 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:05.409271 11041 tablet_bootstrap.cc:654] T bf144ab1a6464a4ab02ac0642c5255e3 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:05.411880 10365 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_aed2460280354a9b94c52d37815f3347" } modify_external_catalogs: true
I20260924 00:21:05.412416 10365 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_aed2460280354a9b94c52d37815f3347" } modify_external_catalogs: true
I20260924 00:21:05.408895 10795 raft_consensus.cc:2454] T 6ac9c7da7fb546f6ae935371799cbad6 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 1.
I20260924 00:21:05.413676 11049 raft_consensus.cc:2229] T 0d555f059f84478eb805f4ca4534dab4 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:05.414129 11049 raft_consensus.cc:2258] T 0d555f059f84478eb805f4ca4534dab4 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:05.414119 10878 leader_election.cc:302] T 6ac9c7da7fb546f6ae935371799cbad6 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:05.414882 11342 raft_consensus.cc:2781] T 6ac9c7da7fb546f6ae935371799cbad6 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:05.415279 11342 raft_consensus.cc:686] T 6ac9c7da7fb546f6ae935371799cbad6 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Becoming Leader. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Running, Role: LEADER
I20260924 00:21:05.416287 11049 ts_tablet_manager.cc:1905] T 0d555f059f84478eb805f4ca4534dab4 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:05.416788 11342 consensus_queue.cc:237] T 6ac9c7da7fb546f6ae935371799cbad6 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:05.418090 10909 tablet_service.cc:1467] Processing CreateTablet for tablet b1c2eb7623f847a3a5b6964627ab94df (DEFAULT_TABLE table=default.Table_aed2460280354a9b94c52d37815f3347 [id=32818e416d4a4677958734f2e7cc2047]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:05.419265 10909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1c2eb7623f847a3a5b6964627ab94df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:05.423436 11323 leader_election.cc:288] T 6ac9c7da7fb546f6ae935371799cbad6 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:05.427054 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet b06bf1e4691441769e162aa5404d4656 (table default.Table_6a2c714f26254ca89f7021633fc89fb7 [id=534f5d0c2e6e426e834efb9b089e4651]) successfully deleted
I20260924 00:21:05.428658 11041 tablet_bootstrap.cc:492] T bf144ab1a6464a4ab02ac0642c5255e3 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:05.429179 11041 ts_tablet_manager.cc:1397] T bf144ab1a6464a4ab02ac0642c5255e3 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.036s user 0.010s sys 0.000s
W20260924 00:21:05.425143 10876 leader_election.cc:341] T 6ac9c7da7fb546f6ae935371799cbad6 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: 6ac9c7da7fb546f6ae935371799cbad6
I20260924 00:21:05.431285 11041 raft_consensus.cc:348] T bf144ab1a6464a4ab02ac0642c5255e3 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:05.431922 11041 raft_consensus.cc:374] T bf144ab1a6464a4ab02ac0642c5255e3 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:05.432210 11041 raft_consensus.cc:729] T bf144ab1a6464a4ab02ac0642c5255e3 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:05.432883 11041 consensus_queue.cc:260] T bf144ab1a6464a4ab02ac0642c5255e3 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:05.440501 11384 tablet.cc:1722] T 1ec36d59b5114feea144bcb72794136c P c8626e300d0644b0908797ad61bd7cef: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260924 00:21:05.446501 11071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:21:05.447778 11071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:21:05.448323 10370 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_ff66b5b591e74da99ad87f8512cdc082" } new_table_name: "default.Table_bf61c7623556425688b99c1c373e4879" modify_external_catalogs: true
I20260924 00:21:05.454098 11428 tablet.cc:1722] T 1ec36d59b5114feea144bcb72794136c P 48fbd603abe146c585d15196e1b5be08: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260924 00:21:05.459599 10369 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_ef1d6027cdee4592aa6cdd9cd1542f0e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:05.461061 10369 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ef1d6027cdee4592aa6cdd9cd1542f0e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:05.464344 10368 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet cf36d53828eb421d87b52b3e4cb1d748
I20260924 00:21:05.464486 10372 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet b36cbb6520de4df58b88cfa53c20ee7a
I20260924 00:21:05.465766 10372 catalog_manager.cc:6565] ReplaceTablet: tablet b36cbb6520de4df58b88cfa53c20ee7a deleted and replaced by tablet 9e44244181324288ad04afb7096839cb
I20260924 00:21:05.468572 11041 ts_tablet_manager.cc:1428] T bf144ab1a6464a4ab02ac0642c5255e3 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.039s user 0.006s sys 0.000s
I20260924 00:21:05.469455 11041 tablet_bootstrap.cc:492] T 4325b60efc1649c9ba4828d0a098a676 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:05.472776 10626 tablet_service.cc:1514] Processing DeleteTablet for tablet b36cbb6520de4df58b88cfa53c20ee7a with delete_type TABLET_DATA_DELETED (replaced by tablet 9e44244181324288ad04afb7096839cb) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:05.474189 10626 tablet_service.cc:1514] Processing DeleteTablet for tablet cf36d53828eb421d87b52b3e4cb1d748 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:05 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:05.477689 11426 tablet.cc:1722] T f294831873d548e4b9201c1bbb964096 P 48fbd603abe146c585d15196e1b5be08: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260924 00:21:05.485548 10878 consensus_peers.cc:487] T 195b033091b543b58ee08417eae2c0e6 P c8626e300d0644b0908797ad61bd7cef -> Peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Couldn't send request to peer 48fbd603abe146c585d15196e1b5be08. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260924 00:21:05.487275 10907 tablet_service.cc:1514] Processing DeleteTablet for tablet cf36d53828eb421d87b52b3e4cb1d748 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:05 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:05.497996 10908 tablet_service.cc:1514] Processing DeleteTablet for tablet b36cbb6520de4df58b88cfa53c20ee7a with delete_type TABLET_DATA_DELETED (replaced by tablet 9e44244181324288ad04afb7096839cb) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:05.498836 11039 tablet_bootstrap.cc:654] T 0980afc2af174da2a8ca2dd414e9825c P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:05.496178 11430 tablet.cc:1722] T 1ec36d59b5114feea144bcb72794136c P b10a9082a37b46f09d581944914c1670: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260924 00:21:05.521143 10368 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_4c92eecf56284783b041439a866855b8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:05.522612 10368 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4c92eecf56284783b041439a866855b8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:05.524720 11049 ts_tablet_manager.cc:1918] T 0d555f059f84478eb805f4ca4534dab4 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:05.526532 11040 tablet_bootstrap.cc:654] T 772fb3d6180e499daf94ceeb9b26048d P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
W20260924 00:21:05.526692 11425 log.cc:924] Time spent T 6ac9c7da7fb546f6ae935371799cbad6 P c8626e300d0644b0908797ad61bd7cef: Append to log took a long time: real 0.082s user 0.003s sys 0.000s
I20260924 00:21:05.534189 10779 tablet_service.cc:1514] Processing DeleteTablet for tablet cf36d53828eb421d87b52b3e4cb1d748 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:05 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:05.536685 10779 tablet_service.cc:1514] Processing DeleteTablet for tablet b36cbb6520de4df58b88cfa53c20ee7a with delete_type TABLET_DATA_DELETED (replaced by tablet 9e44244181324288ad04afb7096839cb) from {username='slave'} at 127.0.0.1:35996
W20260924 00:21:05.538062 10361 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.124:35345: connect: Connection refused (error 111) [suppressed 7 similar messages]
I20260924 00:21:05.545581 11051 ts_tablet_manager.cc:1918] T 666755ec87bb496db75e43ec9204ec5b P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:05.553102 11049 log.cc:1196] T 0d555f059f84478eb805f4ca4534dab4 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/0d555f059f84478eb805f4ca4534dab4
I20260924 00:21:05.553737 11049 ts_tablet_manager.cc:1939] T 0d555f059f84478eb805f4ca4534dab4 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:05.554411 11049 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 170) took 1242 ms. Trace:
I20260924 00:21:05.554689 11049 rpcz_store.cc:276] 0924 00:21:04.311769 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:04.314888 (+ 3119us) service_pool.cc:227] Handling call
0924 00:21:05.397285 (+1082397us) ts_tablet_manager.cc:1145] Deleting tablet 0d555f059f84478eb805f4ca4534dab4
0924 00:21:05.397322 (+ 37us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:05.480630 (+ 83308us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:05.524518 (+ 43888us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:05.554363 (+ 29845us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":15415,"tablet-delete.queue_time_us":1077663}
I20260924 00:21:05.555861 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 0d555f059f84478eb805f4ca4534dab4 (table default.Table_99da18207fc542fbb68972724c4d410e [id=f75c1da3bd734e92bc105c473236fa9b]) successfully deleted
I20260924 00:21:05.556859 11049 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 166) took 1246 ms. Trace:
I20260924 00:21:05.557160 11049 rpcz_store.cc:276] 0924 00:21:04.310255 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:04.314926 (+ 4671us) service_pool.cc:227] Handling call
0924 00:21:05.556558 (+1241632us) ts_tablet_manager.cc:1145] Deleting tablet 717759eb61674c89b05b8d2b0dc34979
0924 00:21:05.556592 (+ 34us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:05.556815 (+ 223us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1236424}
W20260924 00:21:05.558185 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 717759eb61674c89b05b8d2b0dc34979 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 717759eb61674c89b05b8d2b0dc34979 already in progress: creating tablet
I20260924 00:21:05.558876 11051 log.cc:1196] T 666755ec87bb496db75e43ec9204ec5b P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/666755ec87bb496db75e43ec9204ec5b
I20260924 00:21:05.561445 11049 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 168) took 1250 ms. Trace:
I20260924 00:21:05.559898 11051 ts_tablet_manager.cc:1939] T 666755ec87bb496db75e43ec9204ec5b P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:05.561722 11049 rpcz_store.cc:276] 0924 00:21:04.311178 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:04.314863 (+ 3685us) service_pool.cc:227] Handling call
0924 00:21:05.561144 (+1246281us) ts_tablet_manager.cc:1145] Deleting tablet a0f71049f6b3478da3b07ebc9293ce64
0924 00:21:05.561181 (+ 37us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:05.561401 (+ 220us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1240345}
W20260924 00:21:05.562747 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet a0f71049f6b3478da3b07ebc9293ce64 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a0f71049f6b3478da3b07ebc9293ce64 already in progress: creating tablet
I20260924 00:21:05.552717 11041 tablet_bootstrap.cc:654] T 4325b60efc1649c9ba4828d0a098a676 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:05.565264 11039 tablet_bootstrap.cc:492] T 0980afc2af174da2a8ca2dd414e9825c P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:05.565670 11039 ts_tablet_manager.cc:1397] T 0980afc2af174da2a8ca2dd414e9825c P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.187s user 0.013s sys 0.001s
I20260924 00:21:05.568166 11049 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 167) took 1257 ms. Trace:
I20260924 00:21:05.568461 11049 rpcz_store.cc:276] 0924 00:21:04.310872 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:04.315000 (+ 4128us) service_pool.cc:227] Handling call
0924 00:21:05.567863 (+1252863us) ts_tablet_manager.cc:1145] Deleting tablet 675b9e33edcf4d4ea7fa428b476e3945
0924 00:21:05.567900 (+ 37us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:05.568120 (+ 220us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1245737}
W20260924 00:21:05.569525 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 675b9e33edcf4d4ea7fa428b476e3945 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 675b9e33edcf4d4ea7fa428b476e3945 already in progress: creating tablet
I20260924 00:21:05.570456 10374 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_fe2e015f4c3d44f5a3ed8797af219464" } new_table_name: "default.Table_1c64926dc3894391b7b3776711b0c487" modify_external_catalogs: true
I20260924 00:21:05.574693 10373 master_service.cc:946] ReplaceTablet: received request to replace tablet 4420cead1f284af8958f189557833098 from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:05.577463 11049 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 174) took 1184 ms. Trace:
W20260924 00:21:05.577965 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet cf36d53828eb421d87b52b3e4cb1d748 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cf36d53828eb421d87b52b3e4cb1d748 already in progress: creating tablet
W20260924 00:21:05.578634 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet b36cbb6520de4df58b88cfa53c20ee7a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b36cbb6520de4df58b88cfa53c20ee7a already in progress: creating tablet
I20260924 00:21:05.579485 11040 tablet_bootstrap.cc:492] T 772fb3d6180e499daf94ceeb9b26048d P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:05.579830 11040 ts_tablet_manager.cc:1397] T 772fb3d6180e499daf94ceeb9b26048d P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.218s user 0.010s sys 0.000s
I20260924 00:21:05.577760 11049 rpcz_store.cc:276] 0924 00:21:04.393524 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:04.393635 (+ 111us) service_pool.cc:227] Handling call
0924 00:21:05.577144 (+1183509us) ts_tablet_manager.cc:1145] Deleting tablet 9319f9caadb04067a2e9700b5019ed37
0924 00:21:05.577182 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:05.577408 (+ 226us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1182699}
W20260924 00:21:05.581904 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 9319f9caadb04067a2e9700b5019ed37 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9319f9caadb04067a2e9700b5019ed37 already in progress: creating tablet
I20260924 00:21:05.589939 11040 raft_consensus.cc:348] T 772fb3d6180e499daf94ceeb9b26048d P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:05.590590 11040 raft_consensus.cc:374] T 772fb3d6180e499daf94ceeb9b26048d P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:05.590832 11040 raft_consensus.cc:729] T 772fb3d6180e499daf94ceeb9b26048d P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:05.591370 11040 consensus_queue.cc:260] T 772fb3d6180e499daf94ceeb9b26048d P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:05.601338 11039 raft_consensus.cc:348] T 0980afc2af174da2a8ca2dd414e9825c P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:05.606649 11041 tablet_bootstrap.cc:492] T 4325b60efc1649c9ba4828d0a098a676 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:05.607050 11041 ts_tablet_manager.cc:1397] T 4325b60efc1649c9ba4828d0a098a676 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.138s user 0.008s sys 0.000s
I20260924 00:21:05.607373 11039 raft_consensus.cc:374] T 0980afc2af174da2a8ca2dd414e9825c P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:05.608057 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 666755ec87bb496db75e43ec9204ec5b (table default.Table_1347839140d544c3b4aee063e2efe906 [id=00b8c1d674cc460bb7b2425c55037246]) successfully deleted
I20260924 00:21:05.609738 11039 raft_consensus.cc:729] T 0980afc2af174da2a8ca2dd414e9825c P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
W20260924 00:21:05.609783 10358 catalog_manager.cc:4808] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 59c077eb68794340ac900dbedc7056ee because the tablet was not found. No further retry: Not found: Tablet not found: 59c077eb68794340ac900dbedc7056ee
W20260924 00:21:05.610056 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 0de205104d7c4caf8f8f228e6a0d8c3a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0de205104d7c4caf8f8f228e6a0d8c3a already in progress: creating tablet
W20260924 00:21:05.611763 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 5107b763acf04df88553ba0af2757570 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5107b763acf04df88553ba0af2757570 already in progress: creating tablet
W20260924 00:21:05.612088 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet cf36d53828eb421d87b52b3e4cb1d748 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cf36d53828eb421d87b52b3e4cb1d748 already in progress: creating tablet
W20260924 00:21:05.613394 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet d32a0c74b527451fb723e3851d5d055b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d32a0c74b527451fb723e3851d5d055b already in progress: creating tablet
W20260924 00:21:05.614718 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet c7cb7c6892404c7fa593cdaaac67f800 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c7cb7c6892404c7fa593cdaaac67f800 already in progress: creating tablet
I20260924 00:21:05.612188 11039 consensus_queue.cc:260] T 0980afc2af174da2a8ca2dd414e9825c P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
W20260924 00:21:05.616047 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet b36cbb6520de4df58b88cfa53c20ee7a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b36cbb6520de4df58b88cfa53c20ee7a already in progress: creating tablet
W20260924 00:21:05.616379 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet b36cbb6520de4df58b88cfa53c20ee7a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b36cbb6520de4df58b88cfa53c20ee7a already in progress: creating tablet
I20260924 00:21:05.616659 11041 raft_consensus.cc:348] T 4325b60efc1649c9ba4828d0a098a676 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:05.617306 11041 raft_consensus.cc:374] T 4325b60efc1649c9ba4828d0a098a676 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:05.617657 11041 raft_consensus.cc:729] T 4325b60efc1649c9ba4828d0a098a676 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:05.618238 11041 consensus_queue.cc:260] T 4325b60efc1649c9ba4828d0a098a676 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:05.621021 11041 ts_tablet_manager.cc:1428] T 4325b60efc1649c9ba4828d0a098a676 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.014s user 0.007s sys 0.000s
I20260924 00:21:05.628033 11039 ts_tablet_manager.cc:1428] T 0980afc2af174da2a8ca2dd414e9825c P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.062s user 0.004s sys 0.000s
I20260924 00:21:05.630303 10365 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet b1c2eb7623f847a3a5b6964627ab94df
I20260924 00:21:05.632423 11040 ts_tablet_manager.cc:1428] T 772fb3d6180e499daf94ceeb9b26048d P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.052s user 0.005s sys 0.000s
I20260924 00:21:05.635686 10626 tablet_service.cc:1514] Processing DeleteTablet for tablet b1c2eb7623f847a3a5b6964627ab94df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:05 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:05.636075 10779 tablet_service.cc:1514] Processing DeleteTablet for tablet b1c2eb7623f847a3a5b6964627ab94df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:05 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:05.637064 10908 tablet_service.cc:1514] Processing DeleteTablet for tablet b1c2eb7623f847a3a5b6964627ab94df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:05 UTC) from {username='slave'} at 127.0.0.1:35358
W20260924 00:21:05.638542 10358 catalog_manager.cc:4808] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet b1c2eb7623f847a3a5b6964627ab94df because the tablet was not found. No further retry: Not found: Tablet not found: b1c2eb7623f847a3a5b6964627ab94df
I20260924 00:21:05.639807 11041 tablet_bootstrap.cc:492] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
W20260924 00:21:05.642198 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet b1c2eb7623f847a3a5b6964627ab94df because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b1c2eb7623f847a3a5b6964627ab94df already in progress: creating tablet
I20260924 00:21:05.643049 11039 tablet_bootstrap.cc:492] T ffa011cf8dc446de849bea75a4f20056 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:05.649538 11040 tablet_bootstrap.cc:492] T 195b033091b543b58ee08417eae2c0e6 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:05.654629 11040 tablet_bootstrap.cc:654] T 195b033091b543b58ee08417eae2c0e6 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:05.664849 11039 tablet_bootstrap.cc:654] T ffa011cf8dc446de849bea75a4f20056 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:05.666859 11041 tablet_bootstrap.cc:654] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
W20260924 00:21:05.666967 10370 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5f2a32d5afff458fab79ffa106a98884 consensus state has no leader
I20260924 00:21:05.673768 10370 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_de36c9f702024914965bc6b371eda68e" } modify_external_catalogs: true
I20260924 00:21:05.674304 10370 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_de36c9f702024914965bc6b371eda68e" } modify_external_catalogs: true
W20260924 00:21:05.677621 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet cf36d53828eb421d87b52b3e4cb1d748 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cf36d53828eb421d87b52b3e4cb1d748 already in progress: creating tablet
W20260924 00:21:05.678274 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet b1c2eb7623f847a3a5b6964627ab94df because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b1c2eb7623f847a3a5b6964627ab94df already in progress: creating tablet
W20260924 00:21:05.680317 10358 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet cb51d5c6460049108cd7e1974821e578 (table default.Table_9172ff8a00a54ed581e96de0f0b44c3c [id=ed975cbe1bb04e3194882bed0f4ec0b1], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet cb51d5c6460049108cd7e1974821e578 consensus state has no leader
I20260924 00:21:05.693797 11039 tablet_bootstrap.cc:492] T ffa011cf8dc446de849bea75a4f20056 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:05.694166 11039 ts_tablet_manager.cc:1397] T ffa011cf8dc446de849bea75a4f20056 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.051s user 0.008s sys 0.002s
I20260924 00:21:05.696622 11039 raft_consensus.cc:348] T ffa011cf8dc446de849bea75a4f20056 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:05.697350 11039 raft_consensus.cc:374] T ffa011cf8dc446de849bea75a4f20056 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:05.697744 11039 raft_consensus.cc:729] T ffa011cf8dc446de849bea75a4f20056 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:05.698309 11039 consensus_queue.cc:260] T ffa011cf8dc446de849bea75a4f20056 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
W20260924 00:21:05.710008 10374 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a72742864a6b4299b71757fb308dcadf consensus state has no leader
I20260924 00:21:05.710376 11039 ts_tablet_manager.cc:1428] T ffa011cf8dc446de849bea75a4f20056 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.016s user 0.006s sys 0.000s
I20260924 00:21:05.718364 11039 tablet_bootstrap.cc:492] T 2964cd68d5e94f3780a6a3157bd86983 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:05.719723 11040 tablet_bootstrap.cc:492] T 195b033091b543b58ee08417eae2c0e6 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:05.720099 11040 ts_tablet_manager.cc:1397] T 195b033091b543b58ee08417eae2c0e6 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.071s user 0.006s sys 0.001s
I20260924 00:21:05.722167 11040 raft_consensus.cc:348] T 195b033091b543b58ee08417eae2c0e6 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:05.722900 11040 raft_consensus.cc:374] T 195b033091b543b58ee08417eae2c0e6 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:05.723129 11040 raft_consensus.cc:729] T 195b033091b543b58ee08417eae2c0e6 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:05.723196 10374 master_service.cc:946] ReplaceTablet: received request to replace tablet 0980afc2af174da2a8ca2dd414e9825c from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:05.723735 11040 consensus_queue.cc:260] T 195b033091b543b58ee08417eae2c0e6 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:05.731945 11040 ts_tablet_manager.cc:1428] T 195b033091b543b58ee08417eae2c0e6 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.012s user 0.005s sys 0.001s
I20260924 00:21:05.739071 11041 tablet_bootstrap.cc:492] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:05.739477 11041 ts_tablet_manager.cc:1397] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.100s user 0.010s sys 0.000s
I20260924 00:21:05.741528 11041 raft_consensus.cc:348] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:05.742164 11041 raft_consensus.cc:374] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:05.742398 11041 raft_consensus.cc:729] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:05.748564 11041 consensus_queue.cc:260] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:05.765569 11040 tablet_bootstrap.cc:492] T b06bf1e4691441769e162aa5404d4656 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:05.772678 11323 raft_consensus.cc:482] T 772fb3d6180e499daf94ceeb9b26048d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:05.773154 11323 raft_consensus.cc:504] T 772fb3d6180e499daf94ceeb9b26048d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:05.774940 11323 leader_election.cc:288] T 772fb3d6180e499daf94ceeb9b26048d P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:05.775657 10799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "772fb3d6180e499daf94ceeb9b26048d" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:05.776224 10799 raft_consensus.cc:2454] T 772fb3d6180e499daf94ceeb9b26048d P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
I20260924 00:21:05.776546 10656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "772fb3d6180e499daf94ceeb9b26048d" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:05.777143 10656 raft_consensus.cc:2454] T 772fb3d6180e499daf94ceeb9b26048d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
I20260924 00:21:05.777221 10878 leader_election.cc:302] T 772fb3d6180e499daf94ceeb9b26048d P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:05.779245 11323 raft_consensus.cc:2781] T 772fb3d6180e499daf94ceeb9b26048d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:05.779644 11323 raft_consensus.cc:482] T 772fb3d6180e499daf94ceeb9b26048d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:05.780177 11323 raft_consensus.cc:3037] T 772fb3d6180e499daf94ceeb9b26048d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:05.780431 10368 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_f8be74da4e7043749c326a27f9bc4bdf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:05.781879 10368 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f8be74da4e7043749c326a27f9bc4bdf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:05.782971 11039 tablet_bootstrap.cc:654] T 2964cd68d5e94f3780a6a3157bd86983 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:05.787793 11323 raft_consensus.cc:504] T 772fb3d6180e499daf94ceeb9b26048d P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:05.750499 11041 ts_tablet_manager.cc:1428] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20260924 00:21:05.806085 10799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "772fb3d6180e499daf94ceeb9b26048d" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:21:05.806414 10656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "772fb3d6180e499daf94ceeb9b26048d" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:21:05.806620 10799 raft_consensus.cc:3037] T 772fb3d6180e499daf94ceeb9b26048d P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:05.806910 10656 raft_consensus.cc:3037] T 772fb3d6180e499daf94ceeb9b26048d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:05.807891 11041 tablet_bootstrap.cc:492] T 5e624be739394e6d87d6b3fef2f46302 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:05.812422 11041 tablet_bootstrap.cc:654] T 5e624be739394e6d87d6b3fef2f46302 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:05.812994 10799 raft_consensus.cc:2454] T 772fb3d6180e499daf94ceeb9b26048d P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 1.
I20260924 00:21:05.813295 11040 tablet_bootstrap.cc:654] T b06bf1e4691441769e162aa5404d4656 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:05.814067 10878 leader_election.cc:302] T 772fb3d6180e499daf94ceeb9b26048d P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:05.814741 11342 raft_consensus.cc:2781] T 772fb3d6180e499daf94ceeb9b26048d P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:05.815150 11342 raft_consensus.cc:686] T 772fb3d6180e499daf94ceeb9b26048d P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Becoming Leader. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Running, Role: LEADER
I20260924 00:21:05.815879 11342 consensus_queue.cc:237] T 772fb3d6180e499daf94ceeb9b26048d P c8626e300d0644b0908797ad61bd7cef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:05.818390 11404 raft_consensus.cc:482] T 195b033091b543b58ee08417eae2c0e6 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:05.818840 11404 raft_consensus.cc:504] T 195b033091b543b58ee08417eae2c0e6 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:05.824157 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "195b033091b543b58ee08417eae2c0e6" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:05.820780 11404 leader_election.cc:288] T 195b033091b543b58ee08417eae2c0e6 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:05.835176 10368 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_0488528e57a740c0acd6a71051b64e6a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:05.836757 10368 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0488528e57a740c0acd6a71051b64e6a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:05.838948 10369 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_0179ec39d29c4bad9ed0def94394be47" } modify_external_catalogs: true
I20260924 00:21:05.839463 10369 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_0179ec39d29c4bad9ed0def94394be47" } modify_external_catalogs: true
I20260924 00:21:05.842962 10651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "195b033091b543b58ee08417eae2c0e6" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:05.845034 10733 leader_election.cc:302] T 195b033091b543b58ee08417eae2c0e6 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08; no voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:05.846154 11404 raft_consensus.cc:2726] T 195b033091b543b58ee08417eae2c0e6 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:05.849043 10656 raft_consensus.cc:2454] T 772fb3d6180e499daf94ceeb9b26048d P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 1.
I20260924 00:21:05.863288 10369 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet a53f06276dbb4d9f97a154a8ceb0278d
I20260924 00:21:05.865406 10626 tablet_service.cc:1514] Processing DeleteTablet for tablet a53f06276dbb4d9f97a154a8ceb0278d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:05 UTC) from {username='slave'} at 127.0.0.1:46896
W20260924 00:21:05.866885 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet a53f06276dbb4d9f97a154a8ceb0278d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a53f06276dbb4d9f97a154a8ceb0278d already in progress: creating tablet
I20260924 00:21:05.867201 10779 tablet_service.cc:1514] Processing DeleteTablet for tablet a53f06276dbb4d9f97a154a8ceb0278d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:05 UTC) from {username='slave'} at 127.0.0.1:35996
W20260924 00:21:05.868863 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet a53f06276dbb4d9f97a154a8ceb0278d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a53f06276dbb4d9f97a154a8ceb0278d already in progress: creating tablet
I20260924 00:21:05.871485 11323 leader_election.cc:288] T 772fb3d6180e499daf94ceeb9b26048d P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:05.871220 10365 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_9e6a0e1a9c4f44fba041a24df1d601ac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:05.872604 10365 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9e6a0e1a9c4f44fba041a24df1d601ac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:05.877578 10366 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_58cb628381684ce5a8720b6af35aa08c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:05.878330 11039 tablet_bootstrap.cc:492] T 2964cd68d5e94f3780a6a3157bd86983 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:05.878690 11039 ts_tablet_manager.cc:1397] T 2964cd68d5e94f3780a6a3157bd86983 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.160s user 0.008s sys 0.005s
W20260924 00:21:05.878952 10366 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_58cb628381684ce5a8720b6af35aa08c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:05.881047 11039 raft_consensus.cc:348] T 2964cd68d5e94f3780a6a3157bd86983 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:05.881642 11039 raft_consensus.cc:374] T 2964cd68d5e94f3780a6a3157bd86983 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:05.882848 11039 raft_consensus.cc:729] T 2964cd68d5e94f3780a6a3157bd86983 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:05.886967 10369 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_c960d2a8d86a4ce3ba1dbd2f57d3d8b7" } new_table_name: "default.Table_10ca22fe85d94c4b836dc22d24343797" modify_external_catalogs: true
I20260924 00:21:05.888629 11039 consensus_queue.cc:260] T 2964cd68d5e94f3780a6a3157bd86983 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:05.890903 11039 ts_tablet_manager.cc:1428] T 2964cd68d5e94f3780a6a3157bd86983 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.012s user 0.005s sys 0.000s
I20260924 00:21:05.893090 10909 tablet_service.cc:1514] Processing DeleteTablet for tablet a53f06276dbb4d9f97a154a8ceb0278d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:05 UTC) from {username='slave'} at 127.0.0.1:35358
W20260924 00:21:05.894547 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet a53f06276dbb4d9f97a154a8ceb0278d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a53f06276dbb4d9f97a154a8ceb0278d already in progress: creating tablet
I20260924 00:21:05.895318 10367 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 5b0345252c3c496db2987c30adce3da7 (table default.Table_a5d798f7819e403a96f27e98cb0af2bb [id=30a3f7c8fb9c46cf8b883eedae25d938]). Expected version 4 got 0
I20260924 00:21:05.894572 10371 catalog_manager.cc:5495] T f77aea0aad5a430f941aaec0a1c19640 P c8626e300d0644b0908797ad61bd7cef reported cstate change: term changed from 0 to 1, leader changed from <none> to c8626e300d0644b0908797ad61bd7cef (127.10.4.67). New cstate: current_term: 1 leader_uuid: "c8626e300d0644b0908797ad61bd7cef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } health_report { overall_health: UNKNOWN } } }
I20260924 00:21:05.895792 11404 raft_consensus.cc:482] T 0980afc2af174da2a8ca2dd414e9825c P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:05.896144 10367 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 0980afc2af174da2a8ca2dd414e9825c (table default.Table_153086de28a74809b8415724f702a78e [id=222d2010881c4b5c9635aa3c28adc72b]). Expected version 2 got 0
I20260924 00:21:05.896175 11404 raft_consensus.cc:504] T 0980afc2af174da2a8ca2dd414e9825c P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:05.896672 10371 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 195b033091b543b58ee08417eae2c0e6 (table default.Table_ca083375f0df4255b8654ed88040dfcd [id=f79ca8d11aa04d5f8f72d58b798ed4ea]). Expected version 1 got 0
I20260924 00:21:05.897931 11404 leader_election.cc:288] T 0980afc2af174da2a8ca2dd414e9825c P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:05.899187 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0980afc2af174da2a8ca2dd414e9825c" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:05.899735 10942 raft_consensus.cc:2454] T 0980afc2af174da2a8ca2dd414e9825c P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 0.
I20260924 00:21:05.900586 10656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0980afc2af174da2a8ca2dd414e9825c" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:05.901157 10656 raft_consensus.cc:2454] T 0980afc2af174da2a8ca2dd414e9825c P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 0.
I20260924 00:21:05.902168 10733 leader_election.cc:302] T 0980afc2af174da2a8ca2dd414e9825c P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670; no voters:
I20260924 00:21:05.903400 11404 raft_consensus.cc:2781] T 0980afc2af174da2a8ca2dd414e9825c P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:05.903697 11404 raft_consensus.cc:482] T 0980afc2af174da2a8ca2dd414e9825c P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:05.903964 11404 raft_consensus.cc:3037] T 0980afc2af174da2a8ca2dd414e9825c P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:05.911795 11404 raft_consensus.cc:504] T 0980afc2af174da2a8ca2dd414e9825c P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:05.912905 11039 tablet_bootstrap.cc:492] T b06bf1e4691441769e162aa5404d4656 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:05.917524 10368 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_ef1d6027cdee4592aa6cdd9cd1542f0e" } new_table_name: "default.Table_4c16876881e6429fbe20b9eb73c68d3f" modify_external_catalogs: true
I20260924 00:21:05.918783 10656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0980afc2af174da2a8ca2dd414e9825c" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:21:05.919170 11041 tablet_bootstrap.cc:492] T 5e624be739394e6d87d6b3fef2f46302 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:05.919289 10656 raft_consensus.cc:3037] T 0980afc2af174da2a8ca2dd414e9825c P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:05.919519 11041 ts_tablet_manager.cc:1397] T 5e624be739394e6d87d6b3fef2f46302 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.112s user 0.006s sys 0.004s
I20260924 00:21:05.920277 11404 leader_election.cc:288] T 0980afc2af174da2a8ca2dd414e9825c P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Requested vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:05.921906 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0980afc2af174da2a8ca2dd414e9825c" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef"
I20260924 00:21:05.922425 10942 raft_consensus.cc:3037] T 0980afc2af174da2a8ca2dd414e9825c P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:05.924417 10779 tablet_service.cc:1467] Processing CreateTablet for tablet 1492ece2346d461ea57096a897daf83f (DEFAULT_TABLE table=default.Table_284132ac952a4806a25df1a18c9ab709 [id=35a96e566cec479aa95da8543a3faf21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:05.925426 10779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1492ece2346d461ea57096a897daf83f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:05.928368 11041 raft_consensus.cc:348] T 5e624be739394e6d87d6b3fef2f46302 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:05.928985 11041 raft_consensus.cc:374] T 5e624be739394e6d87d6b3fef2f46302 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:05.929219 11041 raft_consensus.cc:729] T 5e624be739394e6d87d6b3fef2f46302 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:05.929766 11041 consensus_queue.cc:260] T 5e624be739394e6d87d6b3fef2f46302 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:05.931595 11041 ts_tablet_manager.cc:1428] T 5e624be739394e6d87d6b3fef2f46302 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.012s user 0.003s sys 0.000s
I20260924 00:21:05.935751 10942 raft_consensus.cc:2454] T 0980afc2af174da2a8ca2dd414e9825c P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 1.
I20260924 00:21:05.934155 10656 raft_consensus.cc:2454] T 0980afc2af174da2a8ca2dd414e9825c P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 1.
I20260924 00:21:05.937139 10733 leader_election.cc:302] T 0980afc2af174da2a8ca2dd414e9825c P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670; no voters:
I20260924 00:21:05.938969 11404 raft_consensus.cc:2781] T 0980afc2af174da2a8ca2dd414e9825c P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:05.939554 10909 tablet_service.cc:1467] Processing CreateTablet for tablet 1492ece2346d461ea57096a897daf83f (DEFAULT_TABLE table=default.Table_284132ac952a4806a25df1a18c9ab709 [id=35a96e566cec479aa95da8543a3faf21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:05.940563 10909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1492ece2346d461ea57096a897daf83f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:05.941179 11404 raft_consensus.cc:686] T 0980afc2af174da2a8ca2dd414e9825c P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: Becoming Leader. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Running, Role: LEADER
I20260924 00:21:05.944903 11040 tablet_bootstrap.cc:492] T b06bf1e4691441769e162aa5404d4656 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:05.945302 11040 ts_tablet_manager.cc:1397] T b06bf1e4691441769e162aa5404d4656 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.180s user 0.007s sys 0.001s
I20260924 00:21:05.942709 11404 consensus_queue.cc:237] T 0980afc2af174da2a8ca2dd414e9825c P 48fbd603abe146c585d15196e1b5be08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:05.947276 11039 tablet_bootstrap.cc:654] T b06bf1e4691441769e162aa5404d4656 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:05.952399 10626 tablet_service.cc:1467] Processing CreateTablet for tablet 1492ece2346d461ea57096a897daf83f (DEFAULT_TABLE table=default.Table_284132ac952a4806a25df1a18c9ab709 [id=35a96e566cec479aa95da8543a3faf21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:05.940820 10773 tablet_service.cc:1467] Processing CreateTablet for tablet 79d8dc7403f448df8cb960311eefe110 (DEFAULT_TABLE table=default.Table_e3fed8d230874b52bf545da40daf0c48 [id=2613a8863ddb477498b493dfcc76114a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:05.954471 10773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79d8dc7403f448df8cb960311eefe110. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:05.962474 10908 tablet_service.cc:1467] Processing CreateTablet for tablet 79d8dc7403f448df8cb960311eefe110 (DEFAULT_TABLE table=default.Table_e3fed8d230874b52bf545da40daf0c48 [id=2613a8863ddb477498b493dfcc76114a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:05.963192 10365 master_service.cc:946] ReplaceTablet: received request to replace tablet 717759eb61674c89b05b8d2b0dc34979 from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:05.963500 10908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79d8dc7403f448df8cb960311eefe110. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:05.963797 11039 tablet_bootstrap.cc:492] T b06bf1e4691441769e162aa5404d4656 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:05.964154 11039 ts_tablet_manager.cc:1397] T b06bf1e4691441769e162aa5404d4656 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.052s user 0.005s sys 0.007s
I20260924 00:21:05.967164 11039 raft_consensus.cc:348] T b06bf1e4691441769e162aa5404d4656 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:05.967780 11039 raft_consensus.cc:374] T b06bf1e4691441769e162aa5404d4656 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:05.968014 11039 raft_consensus.cc:729] T b06bf1e4691441769e162aa5404d4656 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:05.968545 11039 consensus_queue.cc:260] T b06bf1e4691441769e162aa5404d4656 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:05.977922 11041 tablet_bootstrap.cc:492] T 5f2a32d5afff458fab79ffa106a98884 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:05.981000 10626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1492ece2346d461ea57096a897daf83f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:05.985466 10634 tablet_service.cc:1467] Processing CreateTablet for tablet 79d8dc7403f448df8cb960311eefe110 (DEFAULT_TABLE table=default.Table_e3fed8d230874b52bf545da40daf0c48 [id=2613a8863ddb477498b493dfcc76114a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:05.986085 11039 ts_tablet_manager.cc:1428] T b06bf1e4691441769e162aa5404d4656 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.022s user 0.006s sys 0.001s
I20260924 00:21:05.986352 10634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79d8dc7403f448df8cb960311eefe110. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:05.989234 11039 tablet_bootstrap.cc:492] T 119d83562ef546cab496cd7b44fecc15 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:05.989876 11451 raft_consensus.cc:482] T b06bf1e4691441769e162aa5404d4656 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:05.990319 11451 raft_consensus.cc:504] T b06bf1e4691441769e162aa5404d4656 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
W20260924 00:21:05.994380 10876 leader_election.cc:341] T b06bf1e4691441769e162aa5404d4656 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: b06bf1e4691441769e162aa5404d4656
I20260924 00:21:05.994939 11451 leader_election.cc:288] T b06bf1e4691441769e162aa5404d4656 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:05.995294 10799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b06bf1e4691441769e162aa5404d4656" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:05.996762 10878 leader_election.cc:341] T b06bf1e4691441769e162aa5404d4656 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:05.997200 10878 leader_election.cc:302] T b06bf1e4691441769e162aa5404d4656 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670
I20260924 00:21:05.997880 11451 raft_consensus.cc:2726] T b06bf1e4691441769e162aa5404d4656 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:06.001868 10365 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_3f26bb58c6374373a24eb69faf5d68c6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:06.003507 10365 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3f26bb58c6374373a24eb69faf5d68c6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:06.005643 11404 raft_consensus.cc:482] T f77aea0aad5a430f941aaec0a1c19640 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:06.006471 11404 raft_consensus.cc:504] T f77aea0aad5a430f941aaec0a1c19640 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:06.009068 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f77aea0aad5a430f941aaec0a1c19640" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:06.013151 10656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f77aea0aad5a430f941aaec0a1c19640" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:06.013823 10656 raft_consensus.cc:2379] T f77aea0aad5a430f941aaec0a1c19640 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48fbd603abe146c585d15196e1b5be08 in current term 1: Already voted for candidate c8626e300d0644b0908797ad61bd7cef in this term.
W20260924 00:21:06.027009 10369 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 72d3bb80e1034881b94eb75de5d900c2 consensus state has no leader
I20260924 00:21:06.028098 10733 leader_election.cc:302] T f77aea0aad5a430f941aaec0a1c19640 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08; no voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:06.028960 11359 raft_consensus.cc:3037] T f77aea0aad5a430f941aaec0a1c19640 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:06.033645 11041 tablet_bootstrap.cc:654] T 5f2a32d5afff458fab79ffa106a98884 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:06.039325 10372 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet 0980afc2af174da2a8ca2dd414e9825c (table default.Table_153086de28a74809b8415724f702a78e [id=222d2010881c4b5c9635aa3c28adc72b]). Expected version 2 got 0
I20260924 00:21:06.039979 10372 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet 5b0345252c3c496db2987c30adce3da7 (table default.Table_a5d798f7819e403a96f27e98cb0af2bb [id=30a3f7c8fb9c46cf8b883eedae25d938]). Expected version 4 got 0
I20260924 00:21:06.040652 10370 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet f294831873d548e4b9201c1bbb964096
I20260924 00:21:06.043210 10909 tablet_service.cc:1514] Processing DeleteTablet for tablet f294831873d548e4b9201c1bbb964096 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:05 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:06.042786 10626 tablet_service.cc:1514] Processing DeleteTablet for tablet f294831873d548e4b9201c1bbb964096 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:05 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:06.043947 11051 tablet_replica.cc:330] T f294831873d548e4b9201c1bbb964096 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:06.043947 11049 tablet_replica.cc:330] T f294831873d548e4b9201c1bbb964096 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:06.044018 10773 tablet_service.cc:1514] Processing DeleteTablet for tablet f294831873d548e4b9201c1bbb964096 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:05 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:06.044632 11335 tablet_replica.cc:330] T f294831873d548e4b9201c1bbb964096 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:06.044592 11051 raft_consensus.cc:2229] T f294831873d548e4b9201c1bbb964096 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:21:06.044592 11049 raft_consensus.cc:2229] T f294831873d548e4b9201c1bbb964096 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:06.045097 11049 raft_consensus.cc:2258] T f294831873d548e4b9201c1bbb964096 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:06.045387 11051 raft_consensus.cc:2258] T f294831873d548e4b9201c1bbb964096 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:06.045361 11335 raft_consensus.cc:2229] T f294831873d548e4b9201c1bbb964096 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:06.045847 11335 raft_consensus.cc:2258] T f294831873d548e4b9201c1bbb964096 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:06.046403 11359 raft_consensus.cc:2726] T f77aea0aad5a430f941aaec0a1c19640 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:06.046736 11049 ts_tablet_manager.cc:1905] T f294831873d548e4b9201c1bbb964096 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:06.047026 11051 ts_tablet_manager.cc:1905] T f294831873d548e4b9201c1bbb964096 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:06.048877 11335 ts_tablet_manager.cc:1905] T f294831873d548e4b9201c1bbb964096 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:06.052047 11404 leader_election.cc:288] T f77aea0aad5a430f941aaec0a1c19640 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:06.057180 11039 tablet_bootstrap.cc:654] T 119d83562ef546cab496cd7b44fecc15 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:06.069799 11039 tablet_bootstrap.cc:492] T 119d83562ef546cab496cd7b44fecc15 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:06.069583 11040 raft_consensus.cc:348] T b06bf1e4691441769e162aa5404d4656 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:06.070178 11039 ts_tablet_manager.cc:1397] T 119d83562ef546cab496cd7b44fecc15 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.082s user 0.003s sys 0.006s
I20260924 00:21:06.070255 11040 raft_consensus.cc:374] T b06bf1e4691441769e162aa5404d4656 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:06.070499 11040 raft_consensus.cc:729] T b06bf1e4691441769e162aa5404d4656 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:06.071099 11040 consensus_queue.cc:260] T b06bf1e4691441769e162aa5404d4656 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:06.072880 11040 ts_tablet_manager.cc:1428] T b06bf1e4691441769e162aa5404d4656 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.127s user 0.001s sys 0.003s
I20260924 00:21:06.076490 11421 raft_consensus.cc:482] T 4325b60efc1649c9ba4828d0a098a676 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:06.082518 10773 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ac9c7da7fb546f6ae935371799cbad6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:59 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:06.077013 11421 raft_consensus.cc:504] T 4325b60efc1649c9ba4828d0a098a676 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:06.085482 11421 leader_election.cc:288] T 4325b60efc1649c9ba4828d0a098a676 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:06.085620 11049 ts_tablet_manager.cc:1918] T f294831873d548e4b9201c1bbb964096 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:21:06.085963 11049 log.cc:1196] T f294831873d548e4b9201c1bbb964096 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/f294831873d548e4b9201c1bbb964096
I20260924 00:21:06.086608 11049 ts_tablet_manager.cc:1939] T f294831873d548e4b9201c1bbb964096 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:06.087206 10626 tablet_service.cc:1514] Processing DeleteTablet for tablet f3432ac96da94d5099bcf9764c2b32e1 with delete_type TABLET_DATA_DELETED (replaced by tablet 5b43ce1fed0749b8b076b9b026914e07) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:06.088840 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet f294831873d548e4b9201c1bbb964096 (table default.Table_de36c9f702024914965bc6b371eda68e [id=e2cdad6dfe7846ebb3c14f5611cf0f18]) successfully deleted
I20260924 00:21:06.089660 10799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4325b60efc1649c9ba4828d0a098a676" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:06.091166 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4325b60efc1649c9ba4828d0a098a676" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
W20260924 00:21:06.091434 10592 leader_election.cc:341] T 4325b60efc1649c9ba4828d0a098a676 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
W20260924 00:21:06.092470 10589 leader_election.cc:341] T 4325b60efc1649c9ba4828d0a098a676 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:06.092844 10589 leader_election.cc:302] T 4325b60efc1649c9ba4828d0a098a676 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:06.093901 11421 raft_consensus.cc:2726] T 4325b60efc1649c9ba4828d0a098a676 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:06.098923 11041 tablet_bootstrap.cc:492] T 5f2a32d5afff458fab79ffa106a98884 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:06.099388 11041 ts_tablet_manager.cc:1397] T 5f2a32d5afff458fab79ffa106a98884 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.122s user 0.010s sys 0.003s
I20260924 00:21:06.100003 11421 raft_consensus.cc:482] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:06.100445 11421 raft_consensus.cc:504] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:06.103056 10799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95a7a1d2147c4d7ea63ca37a8f0ea4e6" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:06.104485 10592 leader_election.cc:341] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:06.105348 11040 tablet_bootstrap.cc:492] T 2964cd68d5e94f3780a6a3157bd86983 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:06.107440 10907 tablet_service.cc:1467] Processing CreateTablet for tablet c03aa0298c0c4fe580c73cfc61f971a3 (DEFAULT_TABLE table=default.Table_54a305f538ab4d24811b19d3e44e4a90 [id=d1071050336d4b9794f0ce1034da910f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:06.108471 10907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c03aa0298c0c4fe580c73cfc61f971a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:06.108376 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95a7a1d2147c4d7ea63ca37a8f0ea4e6" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
W20260924 00:21:06.109659 10589 leader_election.cc:341] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:06.110059 10589 leader_election.cc:302] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:06.109980 11039 raft_consensus.cc:348] T 119d83562ef546cab496cd7b44fecc15 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:06.110684 11039 raft_consensus.cc:374] T 119d83562ef546cab496cd7b44fecc15 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:06.110915 11039 raft_consensus.cc:729] T 119d83562ef546cab496cd7b44fecc15 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:06.110941 11457 raft_consensus.cc:2726] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:06.111459 11421 leader_election.cc:288] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:06.112989 11041 raft_consensus.cc:348] T 5f2a32d5afff458fab79ffa106a98884 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:06.113663 11041 raft_consensus.cc:374] T 5f2a32d5afff458fab79ffa106a98884 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:06.113896 11041 raft_consensus.cc:729] T 5f2a32d5afff458fab79ffa106a98884 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:06.115054 11041 consensus_queue.cc:260] T 5f2a32d5afff458fab79ffa106a98884 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:06.115906 11323 consensus_queue.cc:1035] T f77aea0aad5a430f941aaec0a1c19640 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Connected to new peer: Peer: permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:06.122061 10634 tablet_service.cc:1514] Processing DeleteTablet for tablet 2964cd68d5e94f3780a6a3157bd86983 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:59 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:06.123034 11039 consensus_queue.cc:260] T 119d83562ef546cab496cd7b44fecc15 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:06.125255 11335 ts_tablet_manager.cc:1918] T f294831873d548e4b9201c1bbb964096 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:21:06.126324 11335 log.cc:1196] T f294831873d548e4b9201c1bbb964096 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/f294831873d548e4b9201c1bbb964096
I20260924 00:21:06.129010 10779 tablet_service.cc:1467] Processing CreateTablet for tablet c03aa0298c0c4fe580c73cfc61f971a3 (DEFAULT_TABLE table=default.Table_54a305f538ab4d24811b19d3e44e4a90 [id=d1071050336d4b9794f0ce1034da910f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:06.124310 11041 ts_tablet_manager.cc:1428] T 5f2a32d5afff458fab79ffa106a98884 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.020s user 0.004s sys 0.000s
I20260924 00:21:06.125289 11049 tablet_replica.cc:330] T f3432ac96da94d5099bcf9764c2b32e1 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:06.131569 10626 tablet_service.cc:1467] Processing CreateTablet for tablet c03aa0298c0c4fe580c73cfc61f971a3 (DEFAULT_TABLE table=default.Table_54a305f538ab4d24811b19d3e44e4a90 [id=d1071050336d4b9794f0ce1034da910f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:06.133522 11457 raft_consensus.cc:482] T 5e624be739394e6d87d6b3fef2f46302 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:06.133781 10626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c03aa0298c0c4fe580c73cfc61f971a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:06.133819 10370 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_6ab0fc6ae11943bea05a803069f3da86" } modify_external_catalogs: true
I20260924 00:21:06.134305 10370 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_6ab0fc6ae11943bea05a803069f3da86" } modify_external_catalogs: true
I20260924 00:21:06.133651 11041 tablet_bootstrap.cc:492] T 18abdec265a24dc8bfc765d4395767c0 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:06.133930 11457 raft_consensus.cc:504] T 5e624be739394e6d87d6b3fef2f46302 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:06.141470 11049 raft_consensus.cc:2229] T f3432ac96da94d5099bcf9764c2b32e1 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:06.141924 11049 raft_consensus.cc:2258] T f3432ac96da94d5099bcf9764c2b32e1 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:06.141729 10369 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_b65a4b1360e74260a30a4d0032e3c025"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:06.142241 11457 leader_election.cc:288] T 5e624be739394e6d87d6b3fef2f46302 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:06.142465 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e624be739394e6d87d6b3fef2f46302" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
W20260924 00:21:06.143114 10369 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b65a4b1360e74260a30a4d0032e3c025 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:06.143424 11049 ts_tablet_manager.cc:1905] T f3432ac96da94d5099bcf9764c2b32e1 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:06.144310 11039 ts_tablet_manager.cc:1428] T 119d83562ef546cab496cd7b44fecc15 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.074s user 0.007s sys 0.000s
W20260924 00:21:06.145092 10589 leader_election.cc:341] T 5e624be739394e6d87d6b3fef2f46302 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:06.147328 10372 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 4325b60efc1649c9ba4828d0a098a676 (table default.Table_4a5edca73a094b4d93ba4d389679ab48 [id=7a76983970cd491a8e8635db379bdd10]). Expected version 3 got 0
I20260924 00:21:06.148113 10372 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 195b033091b543b58ee08417eae2c0e6 (table default.Table_ca083375f0df4255b8654ed88040dfcd [id=f79ca8d11aa04d5f8f72d58b798ed4ea]). Expected version 1 got 0
I20260924 00:21:06.148679 10372 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet bf144ab1a6464a4ab02ac0642c5255e3 (table default.Table_382099cf77a24bb8bd6fe084926431c9 [id=3db896d304514ead8dfe726c402a6fcd]). Expected version 1 got 0
I20260924 00:21:06.156147 10366 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_99f14ba135cc459883dbb7c85dc21e30"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:06.157599 10366 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_99f14ba135cc459883dbb7c85dc21e30 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:06.158679 11039 tablet_bootstrap.cc:492] T 0d555f059f84478eb805f4ca4534dab4 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:06.160111 10373 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_30b352db3bec45a2ac02e7cea36b0d0c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:06.161504 10373 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_30b352db3bec45a2ac02e7cea36b0d0c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260924 00:21:06.166778 10368 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ac8eb7c81dcb4fa099a10d511882c0d4 has no consensus state
I20260924 00:21:06.167145 11039 tablet_bootstrap.cc:654] T 0d555f059f84478eb805f4ca4534dab4 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:06.131377 11335 ts_tablet_manager.cc:1939] T f294831873d548e4b9201c1bbb964096 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:06.187083 11039 tablet_bootstrap.cc:492] T 0d555f059f84478eb805f4ca4534dab4 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:06.187454 11039 ts_tablet_manager.cc:1397] T 0d555f059f84478eb805f4ca4534dab4 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.029s user 0.006s sys 0.005s
I20260924 00:21:06.187170 10773 tablet_service.cc:1467] Processing CreateTablet for tablet d71ce6053fdd4d53b3e0e596148d4c5c (DEFAULT_TABLE table=default.Table_d3365a0130ed438d9a05029476376802 [id=9a2b2f1ebd704a0482022ed1ecb3f332]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:06.188146 10773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d71ce6053fdd4d53b3e0e596148d4c5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:06.144111 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e624be739394e6d87d6b3fef2f46302" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:06.161231 11040 tablet_bootstrap.cc:654] T 2964cd68d5e94f3780a6a3157bd86983 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
W20260924 00:21:06.190104 10592 leader_election.cc:341] T 5e624be739394e6d87d6b3fef2f46302 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:06.190495 10592 leader_election.cc:302] T 5e624be739394e6d87d6b3fef2f46302 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:06.191473 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet f294831873d548e4b9201c1bbb964096 (table default.Table_de36c9f702024914965bc6b371eda68e [id=e2cdad6dfe7846ebb3c14f5611cf0f18]) successfully deleted
I20260924 00:21:06.191776 11457 raft_consensus.cc:2726] T 5e624be739394e6d87d6b3fef2f46302 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:06.192227 11335 tablet_replica.cc:330] T 6ac9c7da7fb546f6ae935371799cbad6 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:06.193146 11335 raft_consensus.cc:2229] T 6ac9c7da7fb546f6ae935371799cbad6 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:06.193676 11335 raft_consensus.cc:2258] T 6ac9c7da7fb546f6ae935371799cbad6 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:06.196255 11335 ts_tablet_manager.cc:1905] T 6ac9c7da7fb546f6ae935371799cbad6 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260924 00:21:06.205505 10876 consensus_peers.cc:487] T 6ac9c7da7fb546f6ae935371799cbad6 P c8626e300d0644b0908797ad61bd7cef -> Peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Couldn't send request to peer b10a9082a37b46f09d581944914c1670. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6ac9c7da7fb546f6ae935371799cbad6. This is attempt 1: this message will repeat every 5th retry.
I20260924 00:21:06.207571 11039 raft_consensus.cc:348] T 0d555f059f84478eb805f4ca4534dab4 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:06.208205 11039 raft_consensus.cc:374] T 0d555f059f84478eb805f4ca4534dab4 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:06.208451 11039 raft_consensus.cc:729] T 0d555f059f84478eb805f4ca4534dab4 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:06.208643 10779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c03aa0298c0c4fe580c73cfc61f971a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:06.209031 11039 consensus_queue.cc:260] T 0d555f059f84478eb805f4ca4534dab4 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:06.221726 10774 tablet_service.cc:1467] Processing CreateTablet for tablet 63e9ebe18f824e18999c6aa5d8286f83 (DEFAULT_TABLE table=default.Table_3879c2e59faa453281d3a65092107295 [id=0742097f4ab743959806cffe991bb5e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:06.222702 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63e9ebe18f824e18999c6aa5d8286f83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:06.228204 10908 tablet_service.cc:1467] Processing CreateTablet for tablet d71ce6053fdd4d53b3e0e596148d4c5c (DEFAULT_TABLE table=default.Table_d3365a0130ed438d9a05029476376802 [id=9a2b2f1ebd704a0482022ed1ecb3f332]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:06.229264 10908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d71ce6053fdd4d53b3e0e596148d4c5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:06.239754 11039 ts_tablet_manager.cc:1428] T 0d555f059f84478eb805f4ca4534dab4 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.052s user 0.006s sys 0.001s
I20260924 00:21:06.240716 11051 ts_tablet_manager.cc:1918] T f294831873d548e4b9201c1bbb964096 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:21:06.241133 11051 log.cc:1196] T f294831873d548e4b9201c1bbb964096 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/f294831873d548e4b9201c1bbb964096
I20260924 00:21:06.242017 11051 ts_tablet_manager.cc:1939] T f294831873d548e4b9201c1bbb964096 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:06.245173 11335 ts_tablet_manager.cc:1918] T 6ac9c7da7fb546f6ae935371799cbad6 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:06.245520 11335 log.cc:1196] T 6ac9c7da7fb546f6ae935371799cbad6 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/6ac9c7da7fb546f6ae935371799cbad6
I20260924 00:21:06.246065 11335 ts_tablet_manager.cc:1939] T 6ac9c7da7fb546f6ae935371799cbad6 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:06.247908 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 6ac9c7da7fb546f6ae935371799cbad6 (table default.Table_b22d604fe0dc4cad9d63baa9ffa5005c [id=9aee3bf701b640cfb7bcba395d4225b4]) successfully deleted
I20260924 00:21:06.248914 10368 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_001f95fae49849f88b710b424fe7714c" } new_table_name: "default.Table_e7d52c1c0aae4f8d841884dc3617ca9c" modify_external_catalogs: true
I20260924 00:21:06.258078 11049 ts_tablet_manager.cc:1918] T f3432ac96da94d5099bcf9764c2b32e1 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:06.258419 11049 log.cc:1196] T f3432ac96da94d5099bcf9764c2b32e1 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/f3432ac96da94d5099bcf9764c2b32e1
I20260924 00:21:06.259012 11049 ts_tablet_manager.cc:1939] T f3432ac96da94d5099bcf9764c2b32e1 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:06.260514 11457 raft_consensus.cc:482] T bf144ab1a6464a4ab02ac0642c5255e3 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:06.260975 11457 raft_consensus.cc:504] T bf144ab1a6464a4ab02ac0642c5255e3 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:06.262811 11457 leader_election.cc:288] T bf144ab1a6464a4ab02ac0642c5255e3 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:06.265012 11041 tablet_bootstrap.cc:654] T 18abdec265a24dc8bfc765d4395767c0 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:06.265215 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf144ab1a6464a4ab02ac0642c5255e3" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:06.266738 10592 leader_election.cc:341] T bf144ab1a6464a4ab02ac0642c5255e3 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:06.269363 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet f3432ac96da94d5099bcf9764c2b32e1 (table default.Table_9608bad42525495eaa4df537b37969e7 [id=aa74d52589984aec8e6d296e5dffd623]) successfully deleted
I20260924 00:21:06.270351 10799 raft_consensus.cc:3037] T 195b033091b543b58ee08417eae2c0e6 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:06.280164 10634 tablet_service.cc:1467] Processing CreateTablet for tablet d71ce6053fdd4d53b3e0e596148d4c5c (DEFAULT_TABLE table=default.Table_d3365a0130ed438d9a05029476376802 [id=9a2b2f1ebd704a0482022ed1ecb3f332]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:06.281349 10634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d71ce6053fdd4d53b3e0e596148d4c5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:06.284179 11421 raft_consensus.cc:482] T 5f2a32d5afff458fab79ffa106a98884 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:06.284474 11049 tablet_replica.cc:330] T 2964cd68d5e94f3780a6a3157bd86983 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:06.285225 11049 raft_consensus.cc:2229] T 2964cd68d5e94f3780a6a3157bd86983 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:06.257294 11039 tablet_bootstrap.cc:492] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:06.283949 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf144ab1a6464a4ab02ac0642c5255e3" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:06.303623 11039 tablet_bootstrap.cc:654] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:06.303448 10909 tablet_service.cc:1467] Processing CreateTablet for tablet 63e9ebe18f824e18999c6aa5d8286f83 (DEFAULT_TABLE table=default.Table_3879c2e59faa453281d3a65092107295 [id=0742097f4ab743959806cffe991bb5e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:06.304778 10909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63e9ebe18f824e18999c6aa5d8286f83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:06.305545 10365 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_1c64926dc3894391b7b3776711b0c487" } modify_external_catalogs: true
I20260924 00:21:06.306041 10365 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_1c64926dc3894391b7b3776711b0c487" } modify_external_catalogs: true
I20260924 00:21:06.307417 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet f294831873d548e4b9201c1bbb964096 (table default.Table_de36c9f702024914965bc6b371eda68e [id=e2cdad6dfe7846ebb3c14f5611cf0f18]) successfully deleted
I20260924 00:21:06.308801 11342 consensus_queue.cc:1035] T f77aea0aad5a430f941aaec0a1c19640 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Connected to new peer: Peer: permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:06.322068 11342 raft_consensus.cc:482] T ffa011cf8dc446de849bea75a4f20056 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:06.322168 11421 raft_consensus.cc:504] T 5f2a32d5afff458fab79ffa106a98884 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:06.322486 11342 raft_consensus.cc:504] T ffa011cf8dc446de849bea75a4f20056 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:06.323928 11421 leader_election.cc:288] T 5f2a32d5afff458fab79ffa106a98884 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:06.324772 11049 raft_consensus.cc:2258] T 2964cd68d5e94f3780a6a3157bd86983 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:06.325331 11342 leader_election.cc:288] T ffa011cf8dc446de849bea75a4f20056 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:06.326509 11049 ts_tablet_manager.cc:1905] T 2964cd68d5e94f3780a6a3157bd86983 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:06.327096 11040 tablet_bootstrap.cc:492] T 2964cd68d5e94f3780a6a3157bd86983 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:06.327435 10798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa011cf8dc446de849bea75a4f20056" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:06.328603 10878 leader_election.cc:341] T ffa011cf8dc446de849bea75a4f20056 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:06.337224 10799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f2a32d5afff458fab79ffa106a98884" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:06.338272 10592 leader_election.cc:341] T 5f2a32d5afff458fab79ffa106a98884 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
W20260924 00:21:06.324668 10589 leader_election.cc:341] T bf144ab1a6464a4ab02ac0642c5255e3 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:06.341302 10589 leader_election.cc:302] T bf144ab1a6464a4ab02ac0642c5255e3 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:06.343775 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f2a32d5afff458fab79ffa106a98884" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
W20260924 00:21:06.350854 10589 leader_election.cc:341] T 5f2a32d5afff458fab79ffa106a98884 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:06.351279 10589 leader_election.cc:302] T 5f2a32d5afff458fab79ffa106a98884 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
W20260924 00:21:06.351588 10876 leader_election.cc:341] T ffa011cf8dc446de849bea75a4f20056 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: ffa011cf8dc446de849bea75a4f20056
I20260924 00:21:06.351981 10876 leader_election.cc:302] T ffa011cf8dc446de849bea75a4f20056 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670
I20260924 00:21:06.352254 11457 raft_consensus.cc:2726] T 5f2a32d5afff458fab79ffa106a98884 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:06.345431 11421 raft_consensus.cc:2726] T bf144ab1a6464a4ab02ac0642c5255e3 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:06.352918 11342 raft_consensus.cc:2726] T ffa011cf8dc446de849bea75a4f20056 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:06.327463 11040 ts_tablet_manager.cc:1397] T 2964cd68d5e94f3780a6a3157bd86983 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.222s user 0.020s sys 0.023s
I20260924 00:21:06.363536 11040 raft_consensus.cc:348] T 2964cd68d5e94f3780a6a3157bd86983 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:06.364192 11040 raft_consensus.cc:374] T 2964cd68d5e94f3780a6a3157bd86983 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:06.364454 11040 raft_consensus.cc:729] T 2964cd68d5e94f3780a6a3157bd86983 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:06.351140 10626 tablet_service.cc:1467] Processing CreateTablet for tablet 63e9ebe18f824e18999c6aa5d8286f83 (DEFAULT_TABLE table=default.Table_3879c2e59faa453281d3a65092107295 [id=0742097f4ab743959806cffe991bb5e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:06.374513 10626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63e9ebe18f824e18999c6aa5d8286f83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:06.375445 10774 tablet_service.cc:1467] Processing CreateTablet for tablet a6e4030e824047ca9cc92d98707d3335 (DEFAULT_TABLE table=default.Table_eff603db4cdf417c871be7eb94681227 [id=ccd80429571145e0bcb1ade6e3a315d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:06.376371 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6e4030e824047ca9cc92d98707d3335. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:06.384629 11039 tablet_bootstrap.cc:492] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:06.386418 11039 ts_tablet_manager.cc:1397] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.129s user 0.015s sys 0.000s
I20260924 00:21:06.389043 10374 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 0980afc2af174da2a8ca2dd414e9825c
I20260924 00:21:06.390381 10374 catalog_manager.cc:6565] ReplaceTablet: tablet 0980afc2af174da2a8ca2dd414e9825c deleted and replaced by tablet a0668cc6cbc547c5bdc6b62a1b38fcde
I20260924 00:21:06.391321 10374 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_58cb628381684ce5a8720b6af35aa08c" } new_table_name: "default.Table_53040e3d85054d8b94a32de64fbedd7e" modify_external_catalogs: true
I20260924 00:21:06.395382 10777 tablet_service.cc:1467] Processing CreateTablet for tablet 8d34535febc94548b48b6b73e1ba85e0 (DEFAULT_TABLE table=default.Table_825c6492de4d43f7ac0521a88ab58441 [id=97c8d87891604e469b0486de66f56a47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:06.396101 10772 tablet_service.cc:1467] Processing CreateTablet for tablet cf0f0f016d074b1fb78d9b427c67fe86 (DEFAULT_TABLE table=default.Table_ff0bbc1b70c2417e9c69a7f7492b2981 [id=667f4349af3f4eb28461e33f65beeb1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:06.396459 10777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d34535febc94548b48b6b73e1ba85e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:06.397083 10772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf0f0f016d074b1fb78d9b427c67fe86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:06.397797 10367 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet 5b0345252c3c496db2987c30adce3da7 (table default.Table_a5d798f7819e403a96f27e98cb0af2bb [id=30a3f7c8fb9c46cf8b883eedae25d938]). Expected version 4 got 0
I20260924 00:21:06.398495 10367 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet 195b033091b543b58ee08417eae2c0e6 (table default.Table_ca083375f0df4255b8654ed88040dfcd [id=f79ca8d11aa04d5f8f72d58b798ed4ea]). Expected version 1 got 0
I20260924 00:21:06.399667 10371 catalog_manager.cc:6040] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: default.Table_66a5eb5613d743c1b96af8b5a252fd85 [id=4dbb68e0a32349ad8da2945db650584d] alter complete (version 2)
W20260924 00:21:06.401607 10371 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.4.67:49569 (request call id 54) took 803 ms (client timeout 1000 ms). Trace:
I20260924 00:21:06.402974 11480 tablet.cc:1722] T 5b0345252c3c496db2987c30adce3da7 P 48fbd603abe146c585d15196e1b5be08: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260924 00:21:06.409162 10773 tablet_service.cc:1514] Processing DeleteTablet for tablet 0980afc2af174da2a8ca2dd414e9825c with delete_type TABLET_DATA_DELETED (replaced by tablet a0668cc6cbc547c5bdc6b62a1b38fcde) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:06.410018 11335 tablet_replica.cc:330] T 0980afc2af174da2a8ca2dd414e9825c P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:06.410723 11335 raft_consensus.cc:2229] T 0980afc2af174da2a8ca2dd414e9825c P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:21:06.411321 11335 pending_rounds.cc:62] T 0980afc2af174da2a8ca2dd414e9825c P 48fbd603abe146c585d15196e1b5be08: Trying to abort 1 pending ops.
I20260924 00:21:06.411504 11335 pending_rounds.cc:68] T 0980afc2af174da2a8ca2dd414e9825c P 48fbd603abe146c585d15196e1b5be08: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332697153327484928 op_type: NO_OP noop_request { }
I20260924 00:21:06.411834 11335 raft_consensus.cc:2866] T 0980afc2af174da2a8ca2dd414e9825c P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260924 00:21:06.412103 11335 raft_consensus.cc:2258] T 0980afc2af174da2a8ca2dd414e9825c P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:06.414465 11335 ts_tablet_manager.cc:1905] T 0980afc2af174da2a8ca2dd414e9825c P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260924 00:21:06.401885 10371 rpcz_store.cc:269] 0924 00:21:05.598510 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:05.598628 (+ 118us) service_pool.cc:227] Handling call
0924 00:21:06.401553 (+802925us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 00:21:05.958026 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 00:21:05.958371 (+ 345us) write_op.cc:432] Acquiring schema lock in shared mode
0924 00:21:05.958394 (+ 23us) write_op.cc:435] Acquired schema lock
0924 00:21:05.958406 (+ 12us) tablet.cc:660] Decoding operations
0924 00:21:05.958640 (+ 234us) write_op.cc:620] Acquiring the partition lock for write op
0924 00:21:05.958672 (+ 32us) write_op.cc:641] Partition lock acquired for write op
0924 00:21:05.958692 (+ 20us) tablet.cc:683] Acquiring locks for 1 operations
0924 00:21:05.958824 (+ 132us) tablet.cc:699] Row locks acquired
0924 00:21:05.958836 (+ 12us) write_op.cc:260] PREPARE: finished
0924 00:21:05.958938 (+ 102us) write_op.cc:270] Start()
0924 00:21:05.959013 (+ 75us) write_op.cc:276] Timestamp: P: 1790209265958829 usec, L: 0
0924 00:21:05.959028 (+ 15us) op_driver.cc:331] REPLICATION: starting
0924 00:21:05.959485 (+ 457us) log.cc:842] Serialized 527 byte log entry
0924 00:21:06.034600 (+ 75115us) op_driver.cc:444] REPLICATION: finished
0924 00:21:06.035182 (+ 582us) write_op.cc:301] APPLY: starting
0924 00:21:06.035237 (+ 55us) tablet.cc:1365] starting BulkCheckPresence
0924 00:21:06.035318 (+ 81us) tablet.cc:1368] finished BulkCheckPresence
0924 00:21:06.035417 (+ 99us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 00:21:06.035573 (+ 156us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 00:21:06.035689 (+ 116us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0924 00:21:06.035729 (+ 40us) write_op.cc:312] APPLY: finished
0924 00:21:06.035884 (+ 155us) log.cc:842] Serialized 23 byte log entry
0924 00:21:06.036026 (+ 142us) write_op.cc:489] Releasing partition, row and schema locks
0924 00:21:06.036085 (+ 59us) write_op.cc:454] Released schema lock
0924 00:21:06.036113 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0924 00:21:06.078261 (+ 42148us) write_op.cc:489] Releasing partition, row and schema locks
0924 00:21:06.079524 (+ 1263us) write_op.cc:454] Released schema lock
Related trace 'op':
0924 00:21:06.253807 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 00:21:06.254134 (+ 327us) write_op.cc:432] Acquiring schema lock in shared mode
0924 00:21:06.254157 (+ 23us) write_op.cc:435] Acquired schema lock
0924 00:21:06.254169 (+ 12us) tablet.cc:660] Decoding operations
0924 00:21:06.254522 (+ 353us) write_op.cc:620] Acquiring the partition lock for write op
0924 00:21:06.254558 (+ 36us) write_op.cc:641] Partition lock acquired for write op
0924 00:21:06.254579 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0924 00:21:06.254713 (+ 134us) tablet.cc:699] Row locks acquired
0924 00:21:06.254725 (+ 12us) write_op.cc:260] PREPARE: finished
0924 00:21:06.254824 (+ 99us) write_op.cc:270] Start()
0924 00:21:06.254906 (+ 82us) write_op.cc:276] Timestamp: P: 1790209266254782 usec, L: 0
0924 00:21:06.254921 (+ 15us) op_driver.cc:331] REPLICATION: starting
0924 00:21:06.255339 (+ 418us) log.cc:842] Serialized 356 byte log entry
0924 00:21:06.302495 (+ 47156us) op_driver.cc:444] REPLICATION: finished
0924 00:21:06.304614 (+ 2119us) write_op.cc:301] APPLY: starting
0924 00:21:06.304658 (+ 44us) tablet.cc:1365] starting BulkCheckPresence
0924 00:21:06.304736 (+ 78us) tablet.cc:1368] finished BulkCheckPresence
0924 00:21:06.304817 (+ 81us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 00:21:06.304970 (+ 153us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 00:21:06.305092 (+ 122us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0924 00:21:06.305134 (+ 42us) write_op.cc:312] APPLY: finished
0924 00:21:06.305282 (+ 148us) log.cc:842] Serialized 23 byte log entry
0924 00:21:06.305397 (+ 115us) write_op.cc:489] Releasing partition, row and schema locks
0924 00:21:06.305449 (+ 52us) write_op.cc:454] Released schema lock
0924 00:21:06.305476 (+ 27us) write_op.cc:341] FINISH: Updating metrics
0924 00:21:06.305627 (+ 151us) write_op.cc:489] Releasing partition, row and schema locks
0924 00:21:06.305651 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":449,"apply.run_cpu_time_us":1392,"apply.run_wall_time_us":44977,"num_ops":1,"prepare.queue_time_us":59839,"prepare.run_cpu_time_us":1845,"prepare.run_wall_time_us":104875,"raft.queue_time_us":50150,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":266,"replication_time_us":75510}],["op",{"apply.queue_time_us":2009,"apply.run_cpu_time_us":1160,"apply.run_wall_time_us":1159,"num_ops":1,"prepare.queue_time_us":215374,"prepare.run_cpu_time_us":1754,"prepare.run_wall_time_us":59345,"raft.queue_time_us":4268,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":211,"replication_time_us":47513}]]}
I20260924 00:21:06.395272 10906 tablet_service.cc:1514] Processing DeleteTablet for tablet 0980afc2af174da2a8ca2dd414e9825c with delete_type TABLET_DATA_DELETED (replaced by tablet a0668cc6cbc547c5bdc6b62a1b38fcde) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:06.396119 10910 tablet_service.cc:1467] Processing CreateTablet for tablet 8d34535febc94548b48b6b73e1ba85e0 (DEFAULT_TABLE table=default.Table_825c6492de4d43f7ac0521a88ab58441 [id=97c8d87891604e469b0486de66f56a47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:06.431814 10910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d34535febc94548b48b6b73e1ba85e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:06.432899 11040 consensus_queue.cc:260] T 2964cd68d5e94f3780a6a3157bd86983 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:06.401558 11451 raft_consensus.cc:482] T 0d555f059f84478eb805f4ca4534dab4 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:06.433955 11451 raft_consensus.cc:504] T 0d555f059f84478eb805f4ca4534dab4 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:06.434891 11040 ts_tablet_manager.cc:1428] T 2964cd68d5e94f3780a6a3157bd86983 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.079s user 0.000s sys 0.004s
I20260924 00:21:06.385470 10907 tablet_service.cc:1467] Processing CreateTablet for tablet a6e4030e824047ca9cc92d98707d3335 (DEFAULT_TABLE table=default.Table_eff603db4cdf417c871be7eb94681227 [id=ccd80429571145e0bcb1ade6e3a315d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:06.435784 11040 tablet_bootstrap.cc:492] T 119d83562ef546cab496cd7b44fecc15 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:06.440722 10906 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ac9c7da7fb546f6ae935371799cbad6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:59 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:06.443151 10772 tablet_service.cc:1514] Processing DeleteTablet for tablet f294831873d548e4b9201c1bbb964096 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:05 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:06.445600 11039 raft_consensus.cc:348] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:06.448333 11039 raft_consensus.cc:374] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:06.448629 11039 raft_consensus.cc:729] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:06.446951 11451 leader_election.cc:288] T 0d555f059f84478eb805f4ca4534dab4 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:21:06.435895 10907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6e4030e824047ca9cc92d98707d3335. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:06.444178 11479 tablet.cc:1722] T 5b0345252c3c496db2987c30adce3da7 P c8626e300d0644b0908797ad61bd7cef: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260924 00:21:06.455668 11039 consensus_queue.cc:260] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:06.457468 11051 tablet_replica.cc:330] T 0980afc2af174da2a8ca2dd414e9825c P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:06.458112 11051 raft_consensus.cc:2229] T 0980afc2af174da2a8ca2dd414e9825c P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:06.458726 10371 master_service.cc:946] ReplaceTablet: received request to replace tablet e0074b1184ec4883ab1016330d98c968 from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:06.459746 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d555f059f84478eb805f4ca4534dab4" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:06.460892 11051 raft_consensus.cc:2258] T 0980afc2af174da2a8ca2dd414e9825c P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:06.461611 11039 ts_tablet_manager.cc:1428] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.031s user 0.005s sys 0.000s
I20260924 00:21:06.462445 11039 tablet_bootstrap.cc:492] T 5e624be739394e6d87d6b3fef2f46302 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:06.462451 11051 ts_tablet_manager.cc:1905] T 0980afc2af174da2a8ca2dd414e9825c P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260924 00:21:06.463587 10878 leader_election.cc:341] T 0d555f059f84478eb805f4ca4534dab4 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:06.470654 10772 tablet_service.cc:1514] Processing DeleteTablet for tablet b06bf1e4691441769e162aa5404d4656 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:59 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:06.491514 10625 tablet_service.cc:1467] Processing CreateTablet for tablet a6e4030e824047ca9cc92d98707d3335 (DEFAULT_TABLE table=default.Table_eff603db4cdf417c871be7eb94681227 [id=ccd80429571145e0bcb1ade6e3a315d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:06.492158 10772 tablet_service.cc:1467] Processing CreateTablet for tablet bc2bbf10bfbc46639f2bf288657ff631 (DEFAULT_TABLE table=default.Table_6ab0fc6ae11943bea05a803069f3da86 [id=338ec36d69914c148a82f36f1a4782c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:06.492569 10625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6e4030e824047ca9cc92d98707d3335. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:06.493156 10772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc2bbf10bfbc46639f2bf288657ff631. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:06.497795 10367 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet 195b033091b543b58ee08417eae2c0e6 (table default.Table_ca083375f0df4255b8654ed88040dfcd [id=f79ca8d11aa04d5f8f72d58b798ed4ea]). Expected version 1 got 0
I20260924 00:21:06.502959 11040 tablet_bootstrap.cc:654] T 119d83562ef546cab496cd7b44fecc15 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:06.503312 10629 tablet_service.cc:1514] Processing DeleteTablet for tablet 0980afc2af174da2a8ca2dd414e9825c with delete_type TABLET_DATA_DELETED (replaced by tablet a0668cc6cbc547c5bdc6b62a1b38fcde) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:06.504453 10632 tablet_service.cc:1467] Processing CreateTablet for tablet 8d34535febc94548b48b6b73e1ba85e0 (DEFAULT_TABLE table=default.Table_825c6492de4d43f7ac0521a88ab58441 [id=97c8d87891604e469b0486de66f56a47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:06.505470 10632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d34535febc94548b48b6b73e1ba85e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:06.508044 11335 ts_tablet_manager.cc:1918] T 0980afc2af174da2a8ca2dd414e9825c P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:06.508354 11335 log.cc:1196] T 0980afc2af174da2a8ca2dd414e9825c P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/0980afc2af174da2a8ca2dd414e9825c
I20260924 00:21:06.509629 10631 tablet_service.cc:1467] Processing CreateTablet for tablet cf0f0f016d074b1fb78d9b427c67fe86 (DEFAULT_TABLE table=default.Table_ff0bbc1b70c2417e9c69a7f7492b2981 [id=667f4349af3f4eb28461e33f65beeb1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:06.510600 10631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf0f0f016d074b1fb78d9b427c67fe86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:06.513139 10635 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e624be739394e6d87d6b3fef2f46302 with delete_type TABLET_DATA_DELETED (replaced by tablet 6bcfd729e9bd4040b3384ab24164e80d) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:06.512982 10369 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_540e09a2364b4d07a38f2015f21fd31e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:06.513693 10630 tablet_service.cc:1514] Processing DeleteTablet for tablet 95a7a1d2147c4d7ea63ca37a8f0ea4e6 with delete_type TABLET_DATA_DELETED (replaced by tablet 3b8d74761b0b4fd0a22021d3870ec05c) from {username='slave'} at 127.0.0.1:46896
W20260924 00:21:06.514374 10369 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_540e09a2364b4d07a38f2015f21fd31e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:06.515026 11335 ts_tablet_manager.cc:1939] T 0980afc2af174da2a8ca2dd414e9825c P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:06.517659 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 0980afc2af174da2a8ca2dd414e9825c (table default.Table_153086de28a74809b8415724f702a78e [id=222d2010881c4b5c9635aa3c28adc72b]) successfully deleted
I20260924 00:21:06.518919 10627 tablet_service.cc:1467] Processing CreateTablet for tablet 5203b62bde2546e5b99034afde9c0813 (DEFAULT_TABLE table=default.Table_a743710df3db4dc7af777801a59305e6 [id=d23bf403ee7d4863bd1e3d6ab879891a]), partition=RANGE (key) PARTITION UNBOUNDED
W20260924 00:21:06.519639 10361 catalog_manager.cc:4808] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet f294831873d548e4b9201c1bbb964096 because the tablet was not found. No further retry: Not found: Tablet not found: f294831873d548e4b9201c1bbb964096
I20260924 00:21:06.519940 10627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5203b62bde2546e5b99034afde9c0813. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:06.520428 11335 tablet_replica.cc:330] T b06bf1e4691441769e162aa5404d4656 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:06.521135 11335 raft_consensus.cc:2229] T b06bf1e4691441769e162aa5404d4656 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:06.521524 11335 raft_consensus.cc:2258] T b06bf1e4691441769e162aa5404d4656 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:06.523015 11335 ts_tablet_manager.cc:1905] T b06bf1e4691441769e162aa5404d4656 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:06.542856 11040 tablet_bootstrap.cc:492] T 119d83562ef546cab496cd7b44fecc15 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:06.543322 11040 ts_tablet_manager.cc:1397] T 119d83562ef546cab496cd7b44fecc15 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.108s user 0.011s sys 0.001s
I20260924 00:21:06.545811 11040 raft_consensus.cc:348] T 119d83562ef546cab496cd7b44fecc15 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:06.546494 11040 raft_consensus.cc:374] T 119d83562ef546cab496cd7b44fecc15 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:06.546732 11040 raft_consensus.cc:729] T 119d83562ef546cab496cd7b44fecc15 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:06.547580 11040 consensus_queue.cc:260] T 119d83562ef546cab496cd7b44fecc15 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:06.605594 11040 ts_tablet_manager.cc:1428] T 119d83562ef546cab496cd7b44fecc15 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.062s user 0.005s sys 0.000s
I20260924 00:21:06.619093 10906 tablet_service.cc:1467] Processing CreateTablet for tablet bc2bbf10bfbc46639f2bf288657ff631 (DEFAULT_TABLE table=default.Table_6ab0fc6ae11943bea05a803069f3da86 [id=338ec36d69914c148a82f36f1a4782c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:06.620216 10906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc2bbf10bfbc46639f2bf288657ff631. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:06.623994 11040 tablet_bootstrap.cc:492] T ffa011cf8dc446de849bea75a4f20056 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
W20260924 00:21:06.625983 10876 leader_election.cc:341] T 0d555f059f84478eb805f4ca4534dab4 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: 0d555f059f84478eb805f4ca4534dab4
I20260924 00:21:06.626412 11041 tablet_bootstrap.cc:492] T 18abdec265a24dc8bfc765d4395767c0 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:06.626847 11041 ts_tablet_manager.cc:1397] T 18abdec265a24dc8bfc765d4395767c0 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.493s user 0.026s sys 0.013s
I20260924 00:21:06.626405 10876 leader_election.cc:302] T 0d555f059f84478eb805f4ca4534dab4 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670
I20260924 00:21:06.628890 11481 raft_consensus.cc:2726] T 0d555f059f84478eb805f4ca4534dab4 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:06.628767 11041 raft_consensus.cc:348] T 18abdec265a24dc8bfc765d4395767c0 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:06.629462 11041 raft_consensus.cc:374] T 18abdec265a24dc8bfc765d4395767c0 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:06.629688 11041 raft_consensus.cc:729] T 18abdec265a24dc8bfc765d4395767c0 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:06.630266 11041 consensus_queue.cc:260] T 18abdec265a24dc8bfc765d4395767c0 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:06.631996 10628 tablet_service.cc:1467] Processing CreateTablet for tablet bc2bbf10bfbc46639f2bf288657ff631 (DEFAULT_TABLE table=default.Table_6ab0fc6ae11943bea05a803069f3da86 [id=338ec36d69914c148a82f36f1a4782c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:06.633252 10628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc2bbf10bfbc46639f2bf288657ff631. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:06.632503 10908 tablet_service.cc:1467] Processing CreateTablet for tablet 5203b62bde2546e5b99034afde9c0813 (DEFAULT_TABLE table=default.Table_a743710df3db4dc7af777801a59305e6 [id=d23bf403ee7d4863bd1e3d6ab879891a]), partition=RANGE (key) PARTITION UNBOUNDED
W20260924 00:21:06.638129 11462 log.cc:924] Time spent T f77aea0aad5a430f941aaec0a1c19640 P b10a9082a37b46f09d581944914c1670: Append to log took a long time: real 0.147s user 0.000s sys 0.009s
W20260924 00:21:06.633260 10878 consensus_peers.cc:487] T 6ac9c7da7fb546f6ae935371799cbad6 P c8626e300d0644b0908797ad61bd7cef -> Peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Couldn't send request to peer 48fbd603abe146c585d15196e1b5be08. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6ac9c7da7fb546f6ae935371799cbad6. This is attempt 1: this message will repeat every 5th retry.
I20260924 00:21:06.636322 11039 tablet_bootstrap.cc:654] T 5e624be739394e6d87d6b3fef2f46302 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:06.649497 10909 tablet_service.cc:1467] Processing CreateTablet for tablet cf0f0f016d074b1fb78d9b427c67fe86 (DEFAULT_TABLE table=default.Table_ff0bbc1b70c2417e9c69a7f7492b2981 [id=667f4349af3f4eb28461e33f65beeb1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:06.657348 10908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5203b62bde2546e5b99034afde9c0813. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:06.659932 11335 ts_tablet_manager.cc:1918] T b06bf1e4691441769e162aa5404d4656 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:06.660264 11335 log.cc:1196] T b06bf1e4691441769e162aa5404d4656 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/b06bf1e4691441769e162aa5404d4656
I20260924 00:21:06.660691 11049 ts_tablet_manager.cc:1918] T 2964cd68d5e94f3780a6a3157bd86983 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:06.661036 11049 log.cc:1196] T 2964cd68d5e94f3780a6a3157bd86983 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/2964cd68d5e94f3780a6a3157bd86983
I20260924 00:21:06.661607 11049 ts_tablet_manager.cc:1939] T 2964cd68d5e94f3780a6a3157bd86983 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:06.661634 10909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf0f0f016d074b1fb78d9b427c67fe86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:06.663992 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 2964cd68d5e94f3780a6a3157bd86983 (table default.Table_89e5d14641754be1934c7e3c880b21ee [id=fd5c26e6faab4fffb49cea70d6ab78a5]) successfully deleted
I20260924 00:21:06.662539 11049 tablet_replica.cc:330] T 0980afc2af174da2a8ca2dd414e9825c P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:06.665455 11049 raft_consensus.cc:2229] T 0980afc2af174da2a8ca2dd414e9825c P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:06.665472 11482 raft_consensus.cc:482] T 119d83562ef546cab496cd7b44fecc15 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:06.665940 11049 raft_consensus.cc:2258] T 0980afc2af174da2a8ca2dd414e9825c P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:06.665875 11482 raft_consensus.cc:504] T 119d83562ef546cab496cd7b44fecc15 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:06.667366 11049 ts_tablet_manager.cc:1905] T 0980afc2af174da2a8ca2dd414e9825c P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:06.667763 11482 leader_election.cc:288] T 119d83562ef546cab496cd7b44fecc15 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:06.669505 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "119d83562ef546cab496cd7b44fecc15" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:06.670123 10797 raft_consensus.cc:2454] T 119d83562ef546cab496cd7b44fecc15 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
I20260924 00:21:06.673240 10878 leader_election.cc:302] T 119d83562ef546cab496cd7b44fecc15 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:06.673601 10779 tablet_service.cc:1467] Processing CreateTablet for tablet 5203b62bde2546e5b99034afde9c0813 (DEFAULT_TABLE table=default.Table_a743710df3db4dc7af777801a59305e6 [id=d23bf403ee7d4863bd1e3d6ab879891a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:06.674645 10779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5203b62bde2546e5b99034afde9c0813. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:06.677699 11040 tablet_bootstrap.cc:654] T ffa011cf8dc446de849bea75a4f20056 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:06.682641 10370 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet bc2bbf10bfbc46639f2bf288657ff631
I20260924 00:21:06.687001 10774 tablet_service.cc:1514] Processing DeleteTablet for tablet bc2bbf10bfbc46639f2bf288657ff631 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:06 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:06.687752 10915 tablet_service.cc:1514] Processing DeleteTablet for tablet bc2bbf10bfbc46639f2bf288657ff631 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:06 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:06.689464 10626 tablet_service.cc:1514] Processing DeleteTablet for tablet bc2bbf10bfbc46639f2bf288657ff631 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:06 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:06.685863 11482 raft_consensus.cc:2781] T 119d83562ef546cab496cd7b44fecc15 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:06.695017 11482 raft_consensus.cc:482] T 119d83562ef546cab496cd7b44fecc15 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:06.660830 11335 ts_tablet_manager.cc:1939] T b06bf1e4691441769e162aa5404d4656 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:06.695475 11482 raft_consensus.cc:3037] T 119d83562ef546cab496cd7b44fecc15 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
W20260924 00:21:06.697026 10360 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 72d3bb80e1034881b94eb75de5d900c2 (table default.Table_10ca22fe85d94c4b836dc22d24343797 [id=ca942906de0f4019bbdf72ec886fe6fb], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 72d3bb80e1034881b94eb75de5d900c2 consensus state has no leader
I20260924 00:21:06.708196 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet b06bf1e4691441769e162aa5404d4656 (table default.Table_6a2c714f26254ca89f7021633fc89fb7 [id=534f5d0c2e6e426e834efb9b089e4651]) successfully deleted
W20260924 00:21:06.713397 10361 catalog_manager.cc:4808] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet bc2bbf10bfbc46639f2bf288657ff631 because the tablet was not found. No further retry: Not found: Tablet not found: bc2bbf10bfbc46639f2bf288657ff631
W20260924 00:21:06.717640 10589 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:41301, user_credentials={real_user=slave}} blocked reactor thread for 89063.8us
I20260924 00:21:06.713749 11041 ts_tablet_manager.cc:1428] T 18abdec265a24dc8bfc765d4395767c0 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.087s user 0.005s sys 0.000s
I20260924 00:21:06.720162 11041 tablet_bootstrap.cc:492] T 4420cead1f284af8958f189557833098 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:06.724295 11040 tablet_bootstrap.cc:492] T ffa011cf8dc446de849bea75a4f20056 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:06.727429 11040 ts_tablet_manager.cc:1397] T ffa011cf8dc446de849bea75a4f20056 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.104s user 0.009s sys 0.000s
W20260924 00:21:06.728673 10368 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a5c560a0b5ce4a1ebb8c0011fe9217bf has no consensus state
I20260924 00:21:06.732182 11040 raft_consensus.cc:348] T ffa011cf8dc446de849bea75a4f20056 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:06.732843 11040 raft_consensus.cc:374] T ffa011cf8dc446de849bea75a4f20056 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:06.733142 11040 raft_consensus.cc:729] T ffa011cf8dc446de849bea75a4f20056 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:06.735564 11040 consensus_queue.cc:260] T ffa011cf8dc446de849bea75a4f20056 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:06.740281 11481 consensus_queue.cc:1035] T 772fb3d6180e499daf94ceeb9b26048d P c8626e300d0644b0908797ad61bd7cef [LEADER]: Connected to new peer: Peer: permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:06.749344 11041 tablet_bootstrap.cc:654] T 4420cead1f284af8958f189557833098 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:06.751870 11040 ts_tablet_manager.cc:1428] T ffa011cf8dc446de849bea75a4f20056 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.024s user 0.005s sys 0.000s
I20260924 00:21:06.755851 11040 tablet_bootstrap.cc:492] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:06.759641 10366 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_db2c1909d90f4836b0b839c96b73a4c2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:06.761040 10366 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_db2c1909d90f4836b0b839c96b73a4c2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:06.774380 11040 tablet_bootstrap.cc:654] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:06.787951 11481 raft_consensus.cc:482] T 2964cd68d5e94f3780a6a3157bd86983 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:06.788419 11481 raft_consensus.cc:504] T 2964cd68d5e94f3780a6a3157bd86983 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:06.789220 11040 tablet_bootstrap.cc:492] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:06.789568 11040 ts_tablet_manager.cc:1397] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.034s user 0.009s sys 0.000s
I20260924 00:21:06.790499 11481 leader_election.cc:288] T 2964cd68d5e94f3780a6a3157bd86983 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:06.791571 11040 raft_consensus.cc:348] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:06.792155 11040 raft_consensus.cc:374] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:06.791985 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2964cd68d5e94f3780a6a3157bd86983" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:06.792555 10795 raft_consensus.cc:2454] T 2964cd68d5e94f3780a6a3157bd86983 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
I20260924 00:21:06.793563 10368 master_service.cc:946] ReplaceTablet: received request to replace tablet 2499754460e04c27ac53d52a4a67f557 from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:06.793726 10878 leader_election.cc:302] T 2964cd68d5e94f3780a6a3157bd86983 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:06.794356 11342 raft_consensus.cc:2781] T 2964cd68d5e94f3780a6a3157bd86983 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:06.794689 11342 raft_consensus.cc:482] T 2964cd68d5e94f3780a6a3157bd86983 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:06.795030 11342 raft_consensus.cc:3037] T 2964cd68d5e94f3780a6a3157bd86983 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:06.804033 11040 raft_consensus.cc:729] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:06.804929 11342 raft_consensus.cc:504] T 2964cd68d5e94f3780a6a3157bd86983 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:06.807648 11342 leader_election.cc:288] T 2964cd68d5e94f3780a6a3157bd86983 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:06.808414 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2964cd68d5e94f3780a6a3157bd86983" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:21:06.808923 10795 raft_consensus.cc:3037] T 2964cd68d5e94f3780a6a3157bd86983 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:06.809429 10651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "119d83562ef546cab496cd7b44fecc15" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:06.810009 10651 raft_consensus.cc:2454] T 119d83562ef546cab496cd7b44fecc15 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
W20260924 00:21:06.814209 10374 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 667e84b0e2d1486289710ea36c594a66 has no consensus state
I20260924 00:21:06.815768 10795 raft_consensus.cc:2454] T 2964cd68d5e94f3780a6a3157bd86983 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 1.
I20260924 00:21:06.817487 10365 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet a72742864a6b4299b71757fb308dcadf
I20260924 00:21:06.818406 10371 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet e0074b1184ec4883ab1016330d98c968
I20260924 00:21:06.816751 10878 leader_election.cc:302] T 2964cd68d5e94f3780a6a3157bd86983 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:06.819491 10371 catalog_manager.cc:6565] ReplaceTablet: tablet e0074b1184ec4883ab1016330d98c968 deleted and replaced by tablet c1784f15ef0e4aaeb487f1ea9e52e83e
I20260924 00:21:06.819437 11481 raft_consensus.cc:2781] T 2964cd68d5e94f3780a6a3157bd86983 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:06.819833 11481 raft_consensus.cc:686] T 2964cd68d5e94f3780a6a3157bd86983 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Becoming Leader. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Running, Role: LEADER
I20260924 00:21:06.736716 11482 raft_consensus.cc:504] T 119d83562ef546cab496cd7b44fecc15 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:06.821470 11481 consensus_queue.cc:237] T 2964cd68d5e94f3780a6a3157bd86983 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:06.825179 10625 tablet_service.cc:1514] Processing DeleteTablet for tablet e0074b1184ec4883ab1016330d98c968 with delete_type TABLET_DATA_DELETED (replaced by tablet c1784f15ef0e4aaeb487f1ea9e52e83e) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:06.823838 10907 tablet_service.cc:1514] Processing DeleteTablet for tablet a72742864a6b4299b71757fb308dcadf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:06 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:06.826150 10910 tablet_service.cc:1514] Processing DeleteTablet for tablet e0074b1184ec4883ab1016330d98c968 with delete_type TABLET_DATA_DELETED (replaced by tablet c1784f15ef0e4aaeb487f1ea9e52e83e) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:06.806161 11040 consensus_queue.cc:260] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:06.837218 10628 tablet_service.cc:1514] Processing DeleteTablet for tablet a72742864a6b4299b71757fb308dcadf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:06 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:06.837555 11040 ts_tablet_manager.cc:1428] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.048s user 0.006s sys 0.000s
I20260924 00:21:06.843730 11040 tablet_bootstrap.cc:492] T 0d555f059f84478eb805f4ca4534dab4 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:06.849357 11041 tablet_bootstrap.cc:492] T 4420cead1f284af8958f189557833098 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:06.849822 11041 ts_tablet_manager.cc:1397] T 4420cead1f284af8958f189557833098 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.130s user 0.009s sys 0.000s
I20260924 00:21:06.851800 11041 raft_consensus.cc:348] T 4420cead1f284af8958f189557833098 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:06.852444 11041 raft_consensus.cc:374] T 4420cead1f284af8958f189557833098 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:06.852711 11041 raft_consensus.cc:729] T 4420cead1f284af8958f189557833098 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:06.853478 11041 consensus_queue.cc:260] T 4420cead1f284af8958f189557833098 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:06.855312 11041 ts_tablet_manager.cc:1428] T 4420cead1f284af8958f189557833098 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.005s user 0.001s sys 0.002s
I20260924 00:21:06.857542 11039 tablet_bootstrap.cc:492] T 5e624be739394e6d87d6b3fef2f46302 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:06.857918 11039 ts_tablet_manager.cc:1397] T 5e624be739394e6d87d6b3fef2f46302 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.396s user 0.009s sys 0.005s
I20260924 00:21:06.859534 11051 ts_tablet_manager.cc:1918] T 0980afc2af174da2a8ca2dd414e9825c P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:06.859906 11051 log.cc:1196] T 0980afc2af174da2a8ca2dd414e9825c P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/0980afc2af174da2a8ca2dd414e9825c
I20260924 00:21:06.860145 11039 raft_consensus.cc:348] T 5e624be739394e6d87d6b3fef2f46302 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:06.860577 11051 ts_tablet_manager.cc:1939] T 0980afc2af174da2a8ca2dd414e9825c P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:06.862775 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 0980afc2af174da2a8ca2dd414e9825c (table default.Table_153086de28a74809b8415724f702a78e [id=222d2010881c4b5c9635aa3c28adc72b]) successfully deleted
I20260924 00:21:06.865053 10772 tablet_service.cc:1514] Processing DeleteTablet for tablet e0074b1184ec4883ab1016330d98c968 with delete_type TABLET_DATA_DELETED (replaced by tablet c1784f15ef0e4aaeb487f1ea9e52e83e) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:06.865792 11335 tablet_replica.cc:330] T e0074b1184ec4883ab1016330d98c968 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:06.866480 11335 raft_consensus.cc:2229] T e0074b1184ec4883ab1016330d98c968 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:06.866925 11335 raft_consensus.cc:2258] T e0074b1184ec4883ab1016330d98c968 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:06.868469 11335 ts_tablet_manager.cc:1905] T e0074b1184ec4883ab1016330d98c968 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:06.868299 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "119d83562ef546cab496cd7b44fecc15" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:21:06.868808 10797 raft_consensus.cc:3037] T 119d83562ef546cab496cd7b44fecc15 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:06.876096 10779 tablet_service.cc:1514] Processing DeleteTablet for tablet a72742864a6b4299b71757fb308dcadf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:06 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:06.885658 11041 tablet_bootstrap.cc:492] T 8a9443159a334093b4135364057e0c8b P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:06.863664 11049 ts_tablet_manager.cc:1918] T 0980afc2af174da2a8ca2dd414e9825c P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:06.888624 10656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "119d83562ef546cab496cd7b44fecc15" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:21:06.889106 11049 log.cc:1196] T 0980afc2af174da2a8ca2dd414e9825c P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/0980afc2af174da2a8ca2dd414e9825c
I20260924 00:21:06.889211 10656 raft_consensus.cc:3037] T 119d83562ef546cab496cd7b44fecc15 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:06.889276 10369 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_9172ff8a00a54ed581e96de0f0b44c3c" } modify_external_catalogs: true
I20260924 00:21:06.889750 11049 ts_tablet_manager.cc:1939] T 0980afc2af174da2a8ca2dd414e9825c P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:06.889832 10369 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_9172ff8a00a54ed581e96de0f0b44c3c" } modify_external_catalogs: true
I20260924 00:21:06.899062 11040 tablet_bootstrap.cc:654] T 0d555f059f84478eb805f4ca4534dab4 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:06.899802 10656 raft_consensus.cc:2454] T 119d83562ef546cab496cd7b44fecc15 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 1.
I20260924 00:21:06.901876 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 0980afc2af174da2a8ca2dd414e9825c (table default.Table_153086de28a74809b8415724f702a78e [id=222d2010881c4b5c9635aa3c28adc72b]) successfully deleted
I20260924 00:21:06.905272 11049 tablet_replica.cc:330] T 5e624be739394e6d87d6b3fef2f46302 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:06.917582 11049 raft_consensus.cc:2229] T 5e624be739394e6d87d6b3fef2f46302 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:06.919056 11049 raft_consensus.cc:2258] T 5e624be739394e6d87d6b3fef2f46302 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:06.920663 11049 ts_tablet_manager.cc:1905] T 5e624be739394e6d87d6b3fef2f46302 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:06.926873 11502 tablet.cc:1722] T 5b0345252c3c496db2987c30adce3da7 P b10a9082a37b46f09d581944914c1670: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260924 00:21:06.929973 10366 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_42a1cf1e668747c88d6179f35dd0e620"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:06.931443 10366 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_42a1cf1e668747c88d6179f35dd0e620 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:06.931540 10365 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_db2c1909d90f4836b0b839c96b73a4c2" } new_table_name: "default.Table_5bdf50c2b69b4f3cadbb52a11cf10035" modify_external_catalogs: true
I20260924 00:21:06.933429 10371 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_cae9fb8a84b248ac96ee0c3ee9874324"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:06.934335 11041 tablet_bootstrap.cc:654] T 8a9443159a334093b4135364057e0c8b P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
W20260924 00:21:06.934712 10371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cae9fb8a84b248ac96ee0c3ee9874324 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:06.940541 11359 raft_consensus.cc:482] T ffa011cf8dc446de849bea75a4f20056 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:06.940973 11359 raft_consensus.cc:504] T ffa011cf8dc446de849bea75a4f20056 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:06.943557 11359 leader_election.cc:288] T ffa011cf8dc446de849bea75a4f20056 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
W20260924 00:21:06.945856 10365 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f4db89087d404354b4392344715b7e97 has no consensus state
W20260924 00:21:06.947979 10361 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.124:35345, user_credentials={real_user=slave}} blocked reactor thread for 68223.2us
I20260924 00:21:06.860733 11039 raft_consensus.cc:374] T 5e624be739394e6d87d6b3fef2f46302 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:06.956743 11039 raft_consensus.cc:729] T 5e624be739394e6d87d6b3fef2f46302 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:06.957489 11039 consensus_queue.cc:260] T 5e624be739394e6d87d6b3fef2f46302 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:06.917623 11335 ts_tablet_manager.cc:1918] T e0074b1184ec4883ab1016330d98c968 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:21:06.958494 11335 log.cc:1196] T e0074b1184ec4883ab1016330d98c968 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/e0074b1184ec4883ab1016330d98c968
I20260924 00:21:06.958943 11039 ts_tablet_manager.cc:1428] T 5e624be739394e6d87d6b3fef2f46302 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.101s user 0.004s sys 0.000s
I20260924 00:21:06.888171 11482 leader_election.cc:288] T 119d83562ef546cab496cd7b44fecc15 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:06.963311 11335 ts_tablet_manager.cc:1939] T e0074b1184ec4883ab1016330d98c968 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:06.965267 11040 tablet_bootstrap.cc:492] T 0d555f059f84478eb805f4ca4534dab4 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:06.965649 11040 ts_tablet_manager.cc:1397] T 0d555f059f84478eb805f4ca4534dab4 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.124s user 0.008s sys 0.001s
I20260924 00:21:06.967774 10797 raft_consensus.cc:2454] T 119d83562ef546cab496cd7b44fecc15 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 1.
I20260924 00:21:06.862757 11051 tablet_replica.cc:330] T 6ac9c7da7fb546f6ae935371799cbad6 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:06.974538 11041 tablet_bootstrap.cc:492] T 8a9443159a334093b4135364057e0c8b P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:06.974516 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet e0074b1184ec4883ab1016330d98c968 (table default.Table_63927aa1872e41da917d6b92c8acb310 [id=485c42c942b74abbb32bea36d25bc184]) successfully deleted
I20260924 00:21:06.975289 11051 raft_consensus.cc:2229] T 6ac9c7da7fb546f6ae935371799cbad6 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:21:06.974939 11041 ts_tablet_manager.cc:1397] T 8a9443159a334093b4135364057e0c8b P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.089s user 0.006s sys 0.001s
W20260924 00:21:06.976027 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet a72742864a6b4299b71757fb308dcadf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a72742864a6b4299b71757fb308dcadf already in progress: creating tablet
I20260924 00:21:06.976190 11051 pending_rounds.cc:62] T 6ac9c7da7fb546f6ae935371799cbad6 P c8626e300d0644b0908797ad61bd7cef: Trying to abort 1 pending ops.
I20260924 00:21:06.976387 11051 pending_rounds.cc:68] T 6ac9c7da7fb546f6ae935371799cbad6 P c8626e300d0644b0908797ad61bd7cef: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332697151159713792 op_type: NO_OP noop_request { }
I20260924 00:21:06.976722 11051 raft_consensus.cc:2866] T 6ac9c7da7fb546f6ae935371799cbad6 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260924 00:21:06.977062 11051 raft_consensus.cc:2258] T 6ac9c7da7fb546f6ae935371799cbad6 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:06.977748 11041 raft_consensus.cc:348] T 8a9443159a334093b4135364057e0c8b P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:06.978364 11041 raft_consensus.cc:374] T 8a9443159a334093b4135364057e0c8b P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:06.978592 11041 raft_consensus.cc:729] T 8a9443159a334093b4135364057e0c8b P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:06.978981 11051 ts_tablet_manager.cc:1905] T 6ac9c7da7fb546f6ae935371799cbad6 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260924 00:21:06.907191 10876 leader_election.cc:341] T 2964cd68d5e94f3780a6a3157bd86983 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: 2964cd68d5e94f3780a6a3157bd86983
W20260924 00:21:06.981359 10876 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:33461, user_credentials={real_user=slave}} blocked reactor thread for 74197.1us
W20260924 00:21:06.982812 10876 leader_election.cc:341] T 2964cd68d5e94f3780a6a3157bd86983 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: 2964cd68d5e94f3780a6a3157bd86983
I20260924 00:21:06.983886 10876 leader_election.cc:302] T 119d83562ef546cab496cd7b44fecc15 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:06.983798 11481 consensus_queue.cc:1035] T 772fb3d6180e499daf94ceeb9b26048d P c8626e300d0644b0908797ad61bd7cef [LEADER]: Connected to new peer: Peer: permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:06.984625 11342 raft_consensus.cc:2781] T 119d83562ef546cab496cd7b44fecc15 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:06.985131 11342 raft_consensus.cc:686] T 119d83562ef546cab496cd7b44fecc15 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Becoming Leader. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Running, Role: LEADER
I20260924 00:21:06.987967 11342 consensus_queue.cc:237] T 119d83562ef546cab496cd7b44fecc15 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:06.977911 11040 raft_consensus.cc:348] T 0d555f059f84478eb805f4ca4534dab4 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:07.015652 11041 consensus_queue.cc:260] T 8a9443159a334093b4135364057e0c8b P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:07.016708 11040 raft_consensus.cc:374] T 0d555f059f84478eb805f4ca4534dab4 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:07.016999 11040 raft_consensus.cc:729] T 0d555f059f84478eb805f4ca4534dab4 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:07.017560 11040 consensus_queue.cc:260] T 0d555f059f84478eb805f4ca4534dab4 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:07.017880 10366 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_6f4df47aea9c41e498e3eb33abc9b832" } new_table_name: "default.Table_a23f89d2bec746fda3e52abc5a0cea95" modify_external_catalogs: true
I20260924 00:21:07.019583 11040 ts_tablet_manager.cc:1428] T 0d555f059f84478eb805f4ca4534dab4 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.051s user 0.005s sys 0.000s
W20260924 00:21:06.992235 10733 leader_election.cc:341] T ffa011cf8dc446de849bea75a4f20056 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: ffa011cf8dc446de849bea75a4f20056
I20260924 00:21:07.021376 11041 ts_tablet_manager.cc:1428] T 8a9443159a334093b4135364057e0c8b P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.045s user 0.005s sys 0.000s
I20260924 00:21:07.024219 10371 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_d3c4523a9e0644f7b0ac485b5b4801db" } modify_external_catalogs: true
I20260924 00:21:07.024727 10371 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_d3c4523a9e0644f7b0ac485b5b4801db" } modify_external_catalogs: true
I20260924 00:21:07.025368 11041 tablet_bootstrap.cc:492] T 0c583d9ba1414f8ea2e605b4b6b076f8 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:07.039408 11040 tablet_bootstrap.cc:492] T f3432ac96da94d5099bcf9764c2b32e1 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
W20260924 00:21:07.044412 10366 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4e9f00abffe14046bf47a6cae7010690 consensus state has no leader
I20260924 00:21:07.045450 10365 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_69f28290116149ff933a5cf08d920b2f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:07.046852 10365 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_69f28290116149ff933a5cf08d920b2f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:07.050510 11040 tablet_bootstrap.cc:654] T f3432ac96da94d5099bcf9764c2b32e1 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:07.050872 10366 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_aceb5020c4374a028638d9931d6665c9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:07.052271 10366 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aceb5020c4374a028638d9931d6665c9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:07.058171 11049 ts_tablet_manager.cc:1918] T 5e624be739394e6d87d6b3fef2f46302 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:07.058532 11049 log.cc:1196] T 5e624be739394e6d87d6b3fef2f46302 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/5e624be739394e6d87d6b3fef2f46302
I20260924 00:21:07.059140 11049 ts_tablet_manager.cc:1939] T 5e624be739394e6d87d6b3fef2f46302 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:07.060292 11482 raft_consensus.cc:482] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:07.060755 11482 raft_consensus.cc:504] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:07.064069 11482 leader_election.cc:288] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
W20260924 00:21:07.061422 10878 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:35359, user_credentials={real_user=slave}} blocked reactor thread for 43817.6us
I20260924 00:21:07.073347 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 5e624be739394e6d87d6b3fef2f46302 (table default.Table_9670783a92074b639ecb0d6c374811e5 [id=df2a5a74db65431c9ecb750184f64709]) successfully deleted
I20260924 00:21:07.074337 11041 tablet_bootstrap.cc:654] T 0c583d9ba1414f8ea2e605b4b6b076f8 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:07.075745 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7e50ac8b7f947b1bfba8cd1cff0ea7e" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:07.076440 10797 raft_consensus.cc:2454] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
I20260924 00:21:07.077203 11049 tablet_replica.cc:330] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:07.081071 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa011cf8dc446de849bea75a4f20056" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:07.081681 10942 raft_consensus.cc:2454] T ffa011cf8dc446de849bea75a4f20056 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 0.
I20260924 00:21:07.085368 11039 tablet_bootstrap.cc:492] T 5f2a32d5afff458fab79ffa106a98884 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:07.089141 10732 leader_election.cc:302] T ffa011cf8dc446de849bea75a4f20056 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters: b10a9082a37b46f09d581944914c1670
I20260924 00:21:07.090114 11359 raft_consensus.cc:2781] T ffa011cf8dc446de849bea75a4f20056 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:07.090451 11359 raft_consensus.cc:482] T ffa011cf8dc446de849bea75a4f20056 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:07.090727 11359 raft_consensus.cc:3037] T ffa011cf8dc446de849bea75a4f20056 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:07.097800 11359 raft_consensus.cc:504] T ffa011cf8dc446de849bea75a4f20056 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:07.100483 11359 leader_election.cc:288] T ffa011cf8dc446de849bea75a4f20056 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Requested vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:07.102205 11040 tablet_bootstrap.cc:492] T f3432ac96da94d5099bcf9764c2b32e1 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:07.102283 10370 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 5f2a32d5afff458fab79ffa106a98884 (table default.Table_bf61c7623556425688b99c1c373e4879 [id=4f70e833d7f74d91af0faa01ed52f78b]). Expected version 1 got 0
I20260924 00:21:07.102759 11040 ts_tablet_manager.cc:1397] T f3432ac96da94d5099bcf9764c2b32e1 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.064s user 0.012s sys 0.000s
I20260924 00:21:07.103008 10370 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 18abdec265a24dc8bfc765d4395767c0 (table default.Table_a5a83435a7fb47ce8083022c0686b74a [id=a587a50b0c204843a41acacc45c2b150]). Expected version 1 got 0
I20260924 00:21:07.104915 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa011cf8dc446de849bea75a4f20056" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef"
I20260924 00:21:07.105496 10942 raft_consensus.cc:3037] T ffa011cf8dc446de849bea75a4f20056 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:07.109732 10878 leader_election.cc:302] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:07.117918 10942 raft_consensus.cc:2454] T ffa011cf8dc446de849bea75a4f20056 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 1.
I20260924 00:21:07.113634 11040 raft_consensus.cc:348] T f3432ac96da94d5099bcf9764c2b32e1 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:07.119094 11040 raft_consensus.cc:374] T f3432ac96da94d5099bcf9764c2b32e1 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:07.119371 11040 raft_consensus.cc:729] T f3432ac96da94d5099bcf9764c2b32e1 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:07.118999 11482 raft_consensus.cc:2781] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:07.119709 11482 raft_consensus.cc:482] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:07.120072 11482 raft_consensus.cc:3037] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:07.120790 10732 leader_election.cc:302] T ffa011cf8dc446de849bea75a4f20056 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters:
W20260924 00:21:07.121390 10733 leader_election.cc:341] T ffa011cf8dc446de849bea75a4f20056 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: ffa011cf8dc446de849bea75a4f20056
I20260924 00:21:07.121551 11359 raft_consensus.cc:2781] T ffa011cf8dc446de849bea75a4f20056 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:07.122677 11049 raft_consensus.cc:2229] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260924 00:21:07.120847 10876 leader_election.cc:341] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: c7e50ac8b7f947b1bfba8cd1cff0ea7e
I20260924 00:21:07.123116 11049 raft_consensus.cc:2258] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:07.124663 11049 ts_tablet_manager.cc:1905] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:07.125622 10368 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 2499754460e04c27ac53d52a4a67f557
I20260924 00:21:07.125763 11359 raft_consensus.cc:686] T ffa011cf8dc446de849bea75a4f20056 P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: Becoming Leader. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Running, Role: LEADER
I20260924 00:21:07.127087 10628 tablet_service.cc:1514] Processing DeleteTablet for tablet 2499754460e04c27ac53d52a4a67f557 with delete_type TABLET_DATA_DELETED (replaced by tablet 7673597f1afc43198e86929a2939269a) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:07.126825 11359 consensus_queue.cc:237] T ffa011cf8dc446de849bea75a4f20056 P 48fbd603abe146c585d15196e1b5be08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:07.129410 10368 catalog_manager.cc:6565] ReplaceTablet: tablet 2499754460e04c27ac53d52a4a67f557 deleted and replaced by tablet 7673597f1afc43198e86929a2939269a
I20260924 00:21:07.131049 10779 tablet_service.cc:1514] Processing DeleteTablet for tablet 2499754460e04c27ac53d52a4a67f557 with delete_type TABLET_DATA_DELETED (replaced by tablet 7673597f1afc43198e86929a2939269a) from {username='slave'} at 127.0.0.1:35996
W20260924 00:21:07.132529 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 2499754460e04c27ac53d52a4a67f557 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2499754460e04c27ac53d52a4a67f557 already in progress: creating tablet
I20260924 00:21:07.133781 10372 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 5b0345252c3c496db2987c30adce3da7 (table default.Table_a5d798f7819e403a96f27e98cb0af2bb [id=30a3f7c8fb9c46cf8b883eedae25d938]). Expected version 4 got 0
I20260924 00:21:07.133977 10908 tablet_service.cc:1514] Processing DeleteTablet for tablet 2499754460e04c27ac53d52a4a67f557 with delete_type TABLET_DATA_DELETED (replaced by tablet 7673597f1afc43198e86929a2939269a) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:07.134621 10372 catalog_manager.cc:5495] T 772fb3d6180e499daf94ceeb9b26048d P c8626e300d0644b0908797ad61bd7cef reported cstate change: term changed from 0 to 1, leader changed from <none> to c8626e300d0644b0908797ad61bd7cef (127.10.4.67). New cstate: current_term: 1 leader_uuid: "c8626e300d0644b0908797ad61bd7cef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } health_report { overall_health: UNKNOWN } } }
I20260924 00:21:07.136332 10372 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 119d83562ef546cab496cd7b44fecc15 (table default.Table_76ffb857cf034b6f80598edd90dad953 [id=5fb5e1e90c9440f385e7b04520bcba9c]). Expected version 1 got 0
I20260924 00:21:07.140364 11051 ts_tablet_manager.cc:1918] T 6ac9c7da7fb546f6ae935371799cbad6 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:07.140728 11051 log.cc:1196] T 6ac9c7da7fb546f6ae935371799cbad6 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/6ac9c7da7fb546f6ae935371799cbad6
I20260924 00:21:07.141425 11051 ts_tablet_manager.cc:1939] T 6ac9c7da7fb546f6ae935371799cbad6 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:07.142834 11040 consensus_queue.cc:260] T f3432ac96da94d5099bcf9764c2b32e1 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:07.149959 11040 ts_tablet_manager.cc:1428] T f3432ac96da94d5099bcf9764c2b32e1 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.047s user 0.005s sys 0.000s
I20260924 00:21:07.156996 10368 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_1ebdc131b4ce4446b021aca0fd94ae99" } new_table_name: "default.Table_d2a991e0c92e4c73b70bfbfb8bea5d58" modify_external_catalogs: true
I20260924 00:21:07.161852 10369 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet cb51d5c6460049108cd7e1974821e578
I20260924 00:21:07.165730 10779 tablet_service.cc:1514] Processing DeleteTablet for tablet cb51d5c6460049108cd7e1974821e578 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:06 UTC) from {username='slave'} at 127.0.0.1:35996
W20260924 00:21:07.167327 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet cb51d5c6460049108cd7e1974821e578 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cb51d5c6460049108cd7e1974821e578 already in progress: creating tablet
I20260924 00:21:07.167706 10628 tablet_service.cc:1514] Processing DeleteTablet for tablet cb51d5c6460049108cd7e1974821e578 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:06 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:07.168510 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 6ac9c7da7fb546f6ae935371799cbad6 (table default.Table_b22d604fe0dc4cad9d63baa9ffa5005c [id=9aee3bf701b640cfb7bcba395d4225b4]) successfully deleted
W20260924 00:21:07.169145 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet bc2bbf10bfbc46639f2bf288657ff631 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bc2bbf10bfbc46639f2bf288657ff631 already in progress: creating tablet
I20260924 00:21:07.169966 10908 tablet_service.cc:1514] Processing DeleteTablet for tablet cb51d5c6460049108cd7e1974821e578 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:06 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:07.172149 10366 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 18abdec265a24dc8bfc765d4395767c0 (table default.Table_a5a83435a7fb47ce8083022c0686b74a [id=a587a50b0c204843a41acacc45c2b150]). Expected version 1 got 0
I20260924 00:21:07.173841 10367 catalog_manager.cc:6040] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: default.Table_a5d798f7819e403a96f27e98cb0af2bb [id=30a3f7c8fb9c46cf8b883eedae25d938] alter complete (version 4)
I20260924 00:21:07.158649 11039 tablet_bootstrap.cc:654] T 5f2a32d5afff458fab79ffa106a98884 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:07.179059 10779 tablet_service.cc:1514] Processing DeleteTablet for tablet 2964cd68d5e94f3780a6a3157bd86983 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:59 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:07.179711 11335 tablet_replica.cc:330] T 2964cd68d5e94f3780a6a3157bd86983 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:07.180361 11335 raft_consensus.cc:2229] T 2964cd68d5e94f3780a6a3157bd86983 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:07.180904 11335 raft_consensus.cc:2258] T 2964cd68d5e94f3780a6a3157bd86983 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:07.181854 10369 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_9dea9b14b2a04787987acb15887351a3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:07.182551 11335 ts_tablet_manager.cc:1905] T 2964cd68d5e94f3780a6a3157bd86983 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260924 00:21:07.183176 10369 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9dea9b14b2a04787987acb15887351a3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:07.186628 10370 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_ee28da98c4ae45d28bb5728f390a54b9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:07.188056 10370 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ee28da98c4ae45d28bb5728f390a54b9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:07.190614 11039 tablet_bootstrap.cc:492] T 5f2a32d5afff458fab79ffa106a98884 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:07.190994 11039 ts_tablet_manager.cc:1397] T 5f2a32d5afff458fab79ffa106a98884 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.106s user 0.008s sys 0.002s
I20260924 00:21:07.194849 10371 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet f85ac5b0fd2348d88e1076b781424ab3
I20260924 00:21:07.197062 10779 tablet_service.cc:1514] Processing DeleteTablet for tablet f85ac5b0fd2348d88e1076b781424ab3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:07 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:07.197423 10908 tablet_service.cc:1514] Processing DeleteTablet for tablet f85ac5b0fd2348d88e1076b781424ab3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:07 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:07.197865 11040 tablet_bootstrap.cc:492] T 5e624be739394e6d87d6b3fef2f46302 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:07.198653 10625 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a9443159a334093b4135364057e0c8b with delete_type TABLET_DATA_DELETED (replaced by tablet a53f06276dbb4d9f97a154a8ceb0278d) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:07.196192 11039 raft_consensus.cc:348] T 5f2a32d5afff458fab79ffa106a98884 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:07.199235 10634 tablet_service.cc:1514] Processing DeleteTablet for tablet f85ac5b0fd2348d88e1076b781424ab3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:07 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:07.199296 11039 raft_consensus.cc:374] T 5f2a32d5afff458fab79ffa106a98884 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:07.199544 11039 raft_consensus.cc:729] T 5f2a32d5afff458fab79ffa106a98884 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:07.200315 10628 tablet_service.cc:1514] Processing DeleteTablet for tablet 4420cead1f284af8958f189557833098 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:01 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:07.204326 11479 tablet.cc:1722] T 195b033091b543b58ee08417eae2c0e6 P c8626e300d0644b0908797ad61bd7cef: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260924 00:21:07.205163 10909 tablet_service.cc:1514] Processing DeleteTablet for tablet 2964cd68d5e94f3780a6a3157bd86983 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:59 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:07.205794 10906 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d555f059f84478eb805f4ca4534dab4 with delete_type TABLET_DATA_DELETED (Sending initial creation of tablet) from {username='slave'} at 127.0.0.1:35358
W20260924 00:21:07.206200 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet a72742864a6b4299b71757fb308dcadf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a72742864a6b4299b71757fb308dcadf already in progress: creating tablet
I20260924 00:21:07.206408 11051 tablet_replica.cc:330] T e0074b1184ec4883ab1016330d98c968 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:07.207079 11051 raft_consensus.cc:2229] T e0074b1184ec4883ab1016330d98c968 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:21:07.207908 11051 raft_consensus.cc:2258] T e0074b1184ec4883ab1016330d98c968 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 00:21:07.211644 10368 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e937cee22326474598cef5c1d09b59c2 consensus state has no leader
I20260924 00:21:07.221354 10908 tablet_service.cc:1467] Processing CreateTablet for tablet 667e84b0e2d1486289710ea36c594a66 (DEFAULT_TABLE table=default.Table_53040e3d85054d8b94a32de64fbedd7e [id=1c3a915542a1470785614e97b5779149]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.221915 10779 tablet_service.cc:1467] Processing CreateTablet for tablet 667e84b0e2d1486289710ea36c594a66 (DEFAULT_TABLE table=default.Table_53040e3d85054d8b94a32de64fbedd7e [id=1c3a915542a1470785614e97b5779149]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.222925 10779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 667e84b0e2d1486289710ea36c594a66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.228142 10906 tablet_service.cc:1467] Processing CreateTablet for tablet e46f92f2a020402681aa597e57f37c41 (DEFAULT_TABLE table=default.Table_0488528e57a740c0acd6a71051b64e6a [id=ffd2134e7d724a74818b24de88da8580]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.229521 10628 tablet_service.cc:1467] Processing CreateTablet for tablet 667e84b0e2d1486289710ea36c594a66 (DEFAULT_TABLE table=default.Table_53040e3d85054d8b94a32de64fbedd7e [id=1c3a915542a1470785614e97b5779149]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.230559 10628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 667e84b0e2d1486289710ea36c594a66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.232174 11335 ts_tablet_manager.cc:1918] T 2964cd68d5e94f3780a6a3157bd86983 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:07.232508 11335 log.cc:1196] T 2964cd68d5e94f3780a6a3157bd86983 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/2964cd68d5e94f3780a6a3157bd86983
I20260924 00:21:07.233620 10634 tablet_service.cc:1467] Processing CreateTablet for tablet e46f92f2a020402681aa597e57f37c41 (DEFAULT_TABLE table=default.Table_0488528e57a740c0acd6a71051b64e6a [id=ffd2134e7d724a74818b24de88da8580]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.233917 10909 tablet_service.cc:1514] Processing DeleteTablet for tablet b06bf1e4691441769e162aa5404d4656 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:59 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:07.234474 10634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e46f92f2a020402681aa597e57f37c41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.234859 10909 tablet_service.cc:1514] Processing DeleteTablet for tablet 0980afc2af174da2a8ca2dd414e9825c with delete_type TABLET_DATA_DELETED (replaced by tablet a0668cc6cbc547c5bdc6b62a1b38fcde) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:07.235838 10909 tablet_service.cc:1514] Processing DeleteTablet for tablet ffa011cf8dc446de849bea75a4f20056 with delete_type TABLET_DATA_DELETED (replaced by tablet 2499754460e04c27ac53d52a4a67f557) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:07.250231 11039 consensus_queue.cc:260] T 5f2a32d5afff458fab79ffa106a98884 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:07.253774 10366 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_d952b34e7fe64fe494f56df574e61d77" } new_table_name: "default.Table_88117cdd98a74bcfa88b82c3162e7119" modify_external_catalogs: true
I20260924 00:21:07.254006 11039 ts_tablet_manager.cc:1428] T 5f2a32d5afff458fab79ffa106a98884 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.063s user 0.002s sys 0.004s
I20260924 00:21:07.259855 11041 tablet_bootstrap.cc:492] T 0c583d9ba1414f8ea2e605b4b6b076f8 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:07.260277 11041 ts_tablet_manager.cc:1397] T 0c583d9ba1414f8ea2e605b4b6b076f8 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.235s user 0.007s sys 0.000s
I20260924 00:21:07.262440 11041 raft_consensus.cc:348] T 0c583d9ba1414f8ea2e605b4b6b076f8 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:07.263106 11041 raft_consensus.cc:374] T 0c583d9ba1414f8ea2e605b4b6b076f8 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:07.263338 11041 raft_consensus.cc:729] T 0c583d9ba1414f8ea2e605b4b6b076f8 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:07.263990 11041 consensus_queue.cc:260] T 0c583d9ba1414f8ea2e605b4b6b076f8 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:07.265810 10370 master_service.cc:946] ReplaceTablet: received request to replace tablet cd30f52583124543b5d616ef94b31831 from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:07.266776 11482 raft_consensus.cc:504] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
W20260924 00:21:07.274791 10876 leader_election.cc:341] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: c7e50ac8b7f947b1bfba8cd1cff0ea7e
I20260924 00:21:07.276351 11482 leader_election.cc:288] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:07.277339 10370 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_e735211dc38c444da1b3af42f354c740"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:07.279284 10370 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e735211dc38c444da1b3af42f354c740 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:07.282150 10625 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c583d9ba1414f8ea2e605b4b6b076f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:01 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:07.288998 10368 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_6453acb2750442d0a172188dbc89d856"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:07.293685 10368 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6453acb2750442d0a172188dbc89d856 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:07.303259 11041 ts_tablet_manager.cc:1428] T 0c583d9ba1414f8ea2e605b4b6b076f8 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.043s user 0.006s sys 0.000s
I20260924 00:21:07.309201 10372 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet 119d83562ef546cab496cd7b44fecc15 (table default.Table_76ffb857cf034b6f80598edd90dad953 [id=5fb5e1e90c9440f385e7b04520bcba9c]). Expected version 1 got 0
I20260924 00:21:07.324141 11041 tablet_bootstrap.cc:492] T a0f71049f6b3478da3b07ebc9293ce64 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:07.330054 10625 tablet_service.cc:1467] Processing CreateTablet for tablet cac5464ea95543b1942c55bdf583db50 (DEFAULT_TABLE table=default.Table_f8be74da4e7043749c326a27f9bc4bdf [id=0b35a077ccef43beba6f4e048b1ca46e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.330317 10909 tablet_service.cc:1467] Processing CreateTablet for tablet cac5464ea95543b1942c55bdf583db50 (DEFAULT_TABLE table=default.Table_f8be74da4e7043749c326a27f9bc4bdf [id=0b35a077ccef43beba6f4e048b1ca46e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.331131 10625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cac5464ea95543b1942c55bdf583db50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.332340 10908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 667e84b0e2d1486289710ea36c594a66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.340680 10906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e46f92f2a020402681aa597e57f37c41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.341305 11039 tablet_bootstrap.cc:492] T f3432ac96da94d5099bcf9764c2b32e1 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:07.345512 10909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cac5464ea95543b1942c55bdf583db50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.351154 11039 tablet_bootstrap.cc:654] T f3432ac96da94d5099bcf9764c2b32e1 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
W20260924 00:21:07.359634 10366 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 60ebab3e93f54383b87aeb54bbde722d consensus state has no leader
I20260924 00:21:07.361572 11457 raft_consensus.cc:482] T 0c583d9ba1414f8ea2e605b4b6b076f8 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:07.362007 11457 raft_consensus.cc:504] T 0c583d9ba1414f8ea2e605b4b6b076f8 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:07.363806 11457 leader_election.cc:288] T 0c583d9ba1414f8ea2e605b4b6b076f8 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:07.365396 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c583d9ba1414f8ea2e605b4b6b076f8" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
W20260924 00:21:07.367422 10589 leader_election.cc:341] T 0c583d9ba1414f8ea2e605b4b6b076f8 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:07.376794 11051 ts_tablet_manager.cc:1905] T e0074b1184ec4883ab1016330d98c968 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:07.377835 11502 tablet.cc:1722] T 195b033091b543b58ee08417eae2c0e6 P b10a9082a37b46f09d581944914c1670: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260924 00:21:07.382920 10368 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_d2a991e0c92e4c73b70bfbfb8bea5d58" } modify_external_catalogs: true
I20260924 00:21:07.384598 11049 ts_tablet_manager.cc:1918] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:07.384961 11049 log.cc:1196] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/95a7a1d2147c4d7ea63ca37a8f0ea4e6
I20260924 00:21:07.385562 11049 ts_tablet_manager.cc:1939] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:07.385926 10368 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_d2a991e0c92e4c73b70bfbfb8bea5d58" } modify_external_catalogs: true
I20260924 00:21:07.387087 11049 tablet_replica.cc:330] T e0074b1184ec4883ab1016330d98c968 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:07.387737 11049 raft_consensus.cc:2229] T e0074b1184ec4883ab1016330d98c968 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:07.388175 11049 raft_consensus.cc:2258] T e0074b1184ec4883ab1016330d98c968 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:07.386749 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 95a7a1d2147c4d7ea63ca37a8f0ea4e6 (table default.Table_0cc93ce7fcfb44089209422666978c62 [id=c0cd199f74494deda9d088345c4c90a3]) successfully deleted
W20260924 00:21:07.391147 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet bc2bbf10bfbc46639f2bf288657ff631 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bc2bbf10bfbc46639f2bf288657ff631 already in progress: creating tablet
I20260924 00:21:07.400220 10370 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 5f2a32d5afff458fab79ffa106a98884 (table default.Table_bf61c7623556425688b99c1c373e4879 [id=4f70e833d7f74d91af0faa01ed52f78b]). Expected version 1 got 0
I20260924 00:21:07.405720 11041 tablet_bootstrap.cc:654] T a0f71049f6b3478da3b07ebc9293ce64 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:07.233085 11335 ts_tablet_manager.cc:1939] T 2964cd68d5e94f3780a6a3157bd86983 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:07.240439 10772 tablet_service.cc:1467] Processing CreateTablet for tablet e46f92f2a020402681aa597e57f37c41 (DEFAULT_TABLE table=default.Table_0488528e57a740c0acd6a71051b64e6a [id=ffd2134e7d724a74818b24de88da8580]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.414709 10772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e46f92f2a020402681aa597e57f37c41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.390884 10777 tablet_service.cc:1467] Processing CreateTablet for tablet 497c7d0c6f1b4a07bbc746f3cab1303d (DEFAULT_TABLE table=default.Table_4c92eecf56284783b041439a866855b8 [id=0311178133ae4be895516123aa4ea4fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.415779 10777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 497c7d0c6f1b4a07bbc746f3cab1303d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.261787 11524 tablet.cc:1722] T 195b033091b543b58ee08417eae2c0e6 P 48fbd603abe146c585d15196e1b5be08: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260924 00:21:07.373817 10773 tablet_service.cc:1514] Processing DeleteTablet for tablet ffa011cf8dc446de849bea75a4f20056 with delete_type TABLET_DATA_DELETED (replaced by tablet 2499754460e04c27ac53d52a4a67f557) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:07.368182 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c583d9ba1414f8ea2e605b4b6b076f8" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:07.421356 10625 tablet_service.cc:1467] Processing CreateTablet for tablet 497c7d0c6f1b4a07bbc746f3cab1303d (DEFAULT_TABLE table=default.Table_4c92eecf56284783b041439a866855b8 [id=0311178133ae4be895516123aa4ea4fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.422192 10370 catalog_manager.cc:5495] T 119d83562ef546cab496cd7b44fecc15 P c8626e300d0644b0908797ad61bd7cef reported cstate change: term changed from 0 to 1, leader changed from <none> to c8626e300d0644b0908797ad61bd7cef (127.10.4.67). New cstate: current_term: 1 leader_uuid: "c8626e300d0644b0908797ad61bd7cef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } health_report { overall_health: HEALTHY } } }
W20260924 00:21:07.422371 10592 leader_election.cc:341] T 0c583d9ba1414f8ea2e605b4b6b076f8 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:07.423693 10592 leader_election.cc:302] T 0c583d9ba1414f8ea2e605b4b6b076f8 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:07.406354 10369 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_0cc93ce7fcfb44089209422666978c62" } modify_external_catalogs: true
I20260924 00:21:07.414906 10371 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_f19e1bc3467b4ac9943c4873aba523de" } new_table_name: "default.Table_d50cb52a8cc04b5b8925615d0c9af895" modify_external_catalogs: true
I20260924 00:21:07.424599 10625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 497c7d0c6f1b4a07bbc746f3cab1303d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.424592 11457 raft_consensus.cc:2726] T 0c583d9ba1414f8ea2e605b4b6b076f8 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:07.240439 10774 tablet_service.cc:1467] Processing CreateTablet for tablet cac5464ea95543b1942c55bdf583db50 (DEFAULT_TABLE table=default.Table_f8be74da4e7043749c326a27f9bc4bdf [id=0b35a077ccef43beba6f4e048b1ca46e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.427067 11039 tablet_bootstrap.cc:492] T f3432ac96da94d5099bcf9764c2b32e1 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:07.427147 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cac5464ea95543b1942c55bdf583db50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.427395 10369 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_0cc93ce7fcfb44089209422666978c62" } modify_external_catalogs: true
I20260924 00:21:07.428085 11039 ts_tablet_manager.cc:1397] T f3432ac96da94d5099bcf9764c2b32e1 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.087s user 0.009s sys 0.006s
I20260924 00:21:07.420017 10368 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet e937cee22326474598cef5c1d09b59c2
I20260924 00:21:07.429538 10773 tablet_service.cc:1514] Processing DeleteTablet for tablet e937cee22326474598cef5c1d09b59c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:07 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:07.430198 10910 tablet_service.cc:1514] Processing DeleteTablet for tablet e937cee22326474598cef5c1d09b59c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:07 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:07.272467 11040 tablet_bootstrap.cc:654] T 5e624be739394e6d87d6b3fef2f46302 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:07.430893 10631 tablet_service.cc:1514] Processing DeleteTablet for tablet e937cee22326474598cef5c1d09b59c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:07 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:07.430951 10370 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 119d83562ef546cab496cd7b44fecc15 (table default.Table_76ffb857cf034b6f80598edd90dad953 [id=5fb5e1e90c9440f385e7b04520bcba9c]). Expected version 1 got 0
I20260924 00:21:07.274071 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7e50ac8b7f947b1bfba8cd1cff0ea7e" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:21:07.435056 10795 raft_consensus.cc:3037] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:07.437610 11039 raft_consensus.cc:348] T f3432ac96da94d5099bcf9764c2b32e1 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:07.441891 11039 raft_consensus.cc:374] T f3432ac96da94d5099bcf9764c2b32e1 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:07.442842 11039 raft_consensus.cc:729] T f3432ac96da94d5099bcf9764c2b32e1 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:07.444258 11039 consensus_queue.cc:260] T f3432ac96da94d5099bcf9764c2b32e1 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:07.447067 10773 tablet_service.cc:1467] Processing CreateTablet for tablet a5c560a0b5ce4a1ebb8c0011fe9217bf (DEFAULT_TABLE table=default.Table_e7d52c1c0aae4f8d841884dc3617ca9c [id=775fb1f73d864bac8c0265786727798c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.447973 10773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5c560a0b5ce4a1ebb8c0011fe9217bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.448026 10906 tablet_service.cc:1467] Processing CreateTablet for tablet 497c7d0c6f1b4a07bbc746f3cab1303d (DEFAULT_TABLE table=default.Table_4c92eecf56284783b041439a866855b8 [id=0311178133ae4be895516123aa4ea4fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.448577 10775 tablet_service.cc:1467] Processing CreateTablet for tablet 69061414550b4bd9bf4b2f40a38d8824 (DEFAULT_TABLE table=default.Table_08f63e23539246519606297527c945af [id=d7d5340352b948e3a72b210b1e99cc25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.449436 10775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69061414550b4bd9bf4b2f40a38d8824. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:21:07.454134 10371 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0c5e5e375e414cd8b268e33b0c13b830 consensus state has no leader
I20260924 00:21:07.454136 11039 ts_tablet_manager.cc:1428] T f3432ac96da94d5099bcf9764c2b32e1 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.026s user 0.006s sys 0.000s
I20260924 00:21:07.454618 10906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 497c7d0c6f1b4a07bbc746f3cab1303d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.461402 11041 tablet_bootstrap.cc:492] T a0f71049f6b3478da3b07ebc9293ce64 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:07.461825 11041 ts_tablet_manager.cc:1397] T a0f71049f6b3478da3b07ebc9293ce64 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.138s user 0.008s sys 0.000s
I20260924 00:21:07.464160 10371 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_35151b4e43bd4e3bbf312c8f1c619e91"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:07.449499 10909 tablet_service.cc:1467] Processing CreateTablet for tablet a5c560a0b5ce4a1ebb8c0011fe9217bf (DEFAULT_TABLE table=default.Table_e7d52c1c0aae4f8d841884dc3617ca9c [id=775fb1f73d864bac8c0265786727798c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.469969 10776 tablet_service.cc:1514] Processing DeleteTablet for tablet c7e50ac8b7f947b1bfba8cd1cff0ea7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:59 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:07.470788 10778 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d555f059f84478eb805f4ca4534dab4 with delete_type TABLET_DATA_DELETED (Sending initial creation of tablet) from {username='slave'} at 127.0.0.1:35996
W20260924 00:21:07.465976 10371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_35151b4e43bd4e3bbf312c8f1c619e91 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:07.479665 10780 tablet_service.cc:1514] Processing DeleteTablet for tablet f3432ac96da94d5099bcf9764c2b32e1 with delete_type TABLET_DATA_DELETED (replaced by tablet 5b43ce1fed0749b8b076b9b026914e07) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:07.485009 11335 tablet_replica.cc:330] T ffa011cf8dc446de849bea75a4f20056 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:07.485678 11335 raft_consensus.cc:2229] T ffa011cf8dc446de849bea75a4f20056 P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:21:07.485707 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 2964cd68d5e94f3780a6a3157bd86983 (table default.Table_89e5d14641754be1934c7e3c880b21ee [id=fd5c26e6faab4fffb49cea70d6ab78a5]) successfully deleted
I20260924 00:21:07.486244 11335 pending_rounds.cc:62] T ffa011cf8dc446de849bea75a4f20056 P 48fbd603abe146c585d15196e1b5be08: Trying to abort 1 pending ops.
W20260924 00:21:07.486313 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet f85ac5b0fd2348d88e1076b781424ab3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f85ac5b0fd2348d88e1076b781424ab3 already in progress: creating tablet
I20260924 00:21:07.486447 11335 pending_rounds.cc:68] T ffa011cf8dc446de849bea75a4f20056 P 48fbd603abe146c585d15196e1b5be08: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332697158228439040 op_type: NO_OP noop_request { }
I20260924 00:21:07.486761 11335 raft_consensus.cc:2866] T ffa011cf8dc446de849bea75a4f20056 P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260924 00:21:07.492139 11335 raft_consensus.cc:2258] T ffa011cf8dc446de849bea75a4f20056 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:07.496793 11335 ts_tablet_manager.cc:1905] T ffa011cf8dc446de849bea75a4f20056 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:07.501734 10628 tablet_service.cc:1467] Processing CreateTablet for tablet a5c560a0b5ce4a1ebb8c0011fe9217bf (DEFAULT_TABLE table=default.Table_e7d52c1c0aae4f8d841884dc3617ca9c [id=775fb1f73d864bac8c0265786727798c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.502804 10628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5c560a0b5ce4a1ebb8c0011fe9217bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.511699 10634 tablet_service.cc:1467] Processing CreateTablet for tablet 69061414550b4bd9bf4b2f40a38d8824 (DEFAULT_TABLE table=default.Table_08f63e23539246519606297527c945af [id=d7d5340352b948e3a72b210b1e99cc25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.513772 11039 tablet_bootstrap.cc:492] T 4420cead1f284af8958f189557833098 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:07.512859 10634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69061414550b4bd9bf4b2f40a38d8824. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.514814 11041 raft_consensus.cc:348] T a0f71049f6b3478da3b07ebc9293ce64 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:07.515383 10909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5c560a0b5ce4a1ebb8c0011fe9217bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.515450 11041 raft_consensus.cc:374] T a0f71049f6b3478da3b07ebc9293ce64 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:07.515682 11041 raft_consensus.cc:729] T a0f71049f6b3478da3b07ebc9293ce64 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:07.516251 11041 consensus_queue.cc:260] T a0f71049f6b3478da3b07ebc9293ce64 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:07.525650 10368 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_c6e518c165b3408d80c30626859c2086"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:07.527128 10368 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c6e518c165b3408d80c30626859c2086 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:07.530170 11051 ts_tablet_manager.cc:1918] T e0074b1184ec4883ab1016330d98c968 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:21:07.530840 10795 raft_consensus.cc:2454] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 1.
I20260924 00:21:07.534765 11041 ts_tablet_manager.cc:1428] T a0f71049f6b3478da3b07ebc9293ce64 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.073s user 0.006s sys 0.000s
I20260924 00:21:07.534929 10369 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 3b8d74761b0b4fd0a22021d3870ec05c
I20260924 00:21:07.531859 10878 leader_election.cc:302] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters: b10a9082a37b46f09d581944914c1670
I20260924 00:21:07.536118 11482 raft_consensus.cc:2781] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:07.536525 11482 raft_consensus.cc:686] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Becoming Leader. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Running, Role: LEADER
I20260924 00:21:07.537241 11482 consensus_queue.cc:237] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P c8626e300d0644b0908797ad61bd7cef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:07.540287 10780 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b8d74761b0b4fd0a22021d3870ec05c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:07 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:07.542121 10908 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b8d74761b0b4fd0a22021d3870ec05c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:07 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:07.543256 11342 raft_consensus.cc:482] T 5e624be739394e6d87d6b3fef2f46302 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:07.543635 11342 raft_consensus.cc:504] T 5e624be739394e6d87d6b3fef2f46302 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:07.545392 11342 leader_election.cc:288] T 5e624be739394e6d87d6b3fef2f46302 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:07.547305 11039 tablet_bootstrap.cc:654] T 4420cead1f284af8958f189557833098 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:07.549170 11051 log.cc:1196] T e0074b1184ec4883ab1016330d98c968 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/e0074b1184ec4883ab1016330d98c968
I20260924 00:21:07.549942 10366 catalog_manager.cc:6040] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: default.Table_ca083375f0df4255b8654ed88040dfcd [id=f79ca8d11aa04d5f8f72d58b798ed4ea] alter complete (version 1)
I20260924 00:21:07.550235 11051 ts_tablet_manager.cc:1939] T e0074b1184ec4883ab1016330d98c968 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:07.552086 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e624be739394e6d87d6b3fef2f46302" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:07.553815 10876 leader_election.cc:341] T 5e624be739394e6d87d6b3fef2f46302 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: 5e624be739394e6d87d6b3fef2f46302
W20260924 00:21:07.554590 10878 leader_election.cc:341] T 5e624be739394e6d87d6b3fef2f46302 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:07.555006 10878 leader_election.cc:302] T 5e624be739394e6d87d6b3fef2f46302 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670
I20260924 00:21:07.555928 11342 raft_consensus.cc:2726] T 5e624be739394e6d87d6b3fef2f46302 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:07.561043 10908 tablet_service.cc:1514] Processing DeleteTablet for tablet c7e50ac8b7f947b1bfba8cd1cff0ea7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:20:59 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:07.562973 10908 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e624be739394e6d87d6b3fef2f46302 with delete_type TABLET_DATA_DELETED (replaced by tablet 6bcfd729e9bd4040b3384ab24164e80d) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:07.563009 10366 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_14e49ca823664cc38f51183bc9bdc0ac" } new_table_name: "default.Table_5db49282d679494c84606725e942b198" modify_external_catalogs: true
I20260924 00:21:07.565582 11041 tablet_bootstrap.cc:492] T 60ebab3e93f54383b87aeb54bbde722d P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:07.568289 10908 tablet_service.cc:1467] Processing CreateTablet for tablet 69061414550b4bd9bf4b2f40a38d8824 (DEFAULT_TABLE table=default.Table_08f63e23539246519606297527c945af [id=d7d5340352b948e3a72b210b1e99cc25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.568753 10773 tablet_service.cc:1467] Processing CreateTablet for tablet 811a6a39194f4fb293bc61da1beacb62 (DEFAULT_TABLE table=default.Table_c41522845ef54f468c7686bc6b171fe6 [id=3f1f2d3ebded41a389cf2e4aca0e060e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.569422 10908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69061414550b4bd9bf4b2f40a38d8824. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.569662 10773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 811a6a39194f4fb293bc61da1beacb62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.569962 10631 tablet_service.cc:1467] Processing CreateTablet for tablet 811a6a39194f4fb293bc61da1beacb62 (DEFAULT_TABLE table=default.Table_c41522845ef54f468c7686bc6b171fe6 [id=3f1f2d3ebded41a389cf2e4aca0e060e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.570925 10631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 811a6a39194f4fb293bc61da1beacb62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.571210 10780 tablet_service.cc:1467] Processing CreateTablet for tablet df24a60e710241de9110afc5a6167bf9 (DEFAULT_TABLE table=default.Table_3f26bb58c6374373a24eb69faf5d68c6 [id=c5d285065f184765b488817af0b5d87d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.572037 10780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df24a60e710241de9110afc5a6167bf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.572062 10632 tablet_service.cc:1467] Processing CreateTablet for tablet df24a60e710241de9110afc5a6167bf9 (DEFAULT_TABLE table=default.Table_3f26bb58c6374373a24eb69faf5d68c6 [id=c5d285065f184765b488817af0b5d87d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.572969 10632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df24a60e710241de9110afc5a6167bf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.575562 11051 tablet_replica.cc:330] T 2964cd68d5e94f3780a6a3157bd86983 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:07.582230 10627 tablet_service.cc:1514] Processing DeleteTablet for tablet a0f71049f6b3478da3b07ebc9293ce64 with delete_type TABLET_DATA_DELETED (replaced by tablet d574642ebc6340fcb5e83c36804ee9f8) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:07.583122 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet e0074b1184ec4883ab1016330d98c968 (table default.Table_63927aa1872e41da917d6b92c8acb310 [id=485c42c942b74abbb32bea36d25bc184]) successfully deleted
W20260924 00:21:07.583736 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 2499754460e04c27ac53d52a4a67f557 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2499754460e04c27ac53d52a4a67f557 already in progress: creating tablet
W20260924 00:21:07.584246 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet cb51d5c6460049108cd7e1974821e578 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cb51d5c6460049108cd7e1974821e578 already in progress: creating tablet
I20260924 00:21:07.584604 11335 ts_tablet_manager.cc:1918] T ffa011cf8dc446de849bea75a4f20056 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260924 00:21:07.584776 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet f85ac5b0fd2348d88e1076b781424ab3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f85ac5b0fd2348d88e1076b781424ab3 already in progress: creating tablet
I20260924 00:21:07.584926 11335 log.cc:1196] T ffa011cf8dc446de849bea75a4f20056 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/ffa011cf8dc446de849bea75a4f20056
I20260924 00:21:07.585570 11335 ts_tablet_manager.cc:1939] T ffa011cf8dc446de849bea75a4f20056 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:07.582350 10369 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_4288ed69ffe84f6babc2d310dd022107"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:07.587697 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet ffa011cf8dc446de849bea75a4f20056 (table default.Table_6cf69012192f4f1087e29db70ae8c282 [id=93c1734de58245f8b1d5ab88ec9f0c8e]) successfully deleted
W20260924 00:21:07.587875 10369 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4288ed69ffe84f6babc2d310dd022107 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:07.589401 11051 raft_consensus.cc:2229] T 2964cd68d5e94f3780a6a3157bd86983 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:21:07.590898 10367 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_6ba8cc42d97e4576a8c2021b39b78dbb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:07.592254 10367 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6ba8cc42d97e4576a8c2021b39b78dbb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:07.593900 11049 ts_tablet_manager.cc:1905] T e0074b1184ec4883ab1016330d98c968 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:07.594277 11051 pending_rounds.cc:62] T 2964cd68d5e94f3780a6a3157bd86983 P c8626e300d0644b0908797ad61bd7cef: Trying to abort 1 pending ops.
I20260924 00:21:07.594491 11051 pending_rounds.cc:68] T 2964cd68d5e94f3780a6a3157bd86983 P c8626e300d0644b0908797ad61bd7cef: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332697156915707904 op_type: NO_OP noop_request { }
I20260924 00:21:07.594810 11051 raft_consensus.cc:2866] T 2964cd68d5e94f3780a6a3157bd86983 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260924 00:21:07.595114 11051 raft_consensus.cc:2258] T 2964cd68d5e94f3780a6a3157bd86983 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:07.577483 11342 raft_consensus.cc:482] T 5f2a32d5afff458fab79ffa106a98884 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:07.595602 11040 tablet_bootstrap.cc:492] T 5e624be739394e6d87d6b3fef2f46302 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:07.596202 11040 ts_tablet_manager.cc:1397] T 5e624be739394e6d87d6b3fef2f46302 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.398s user 0.020s sys 0.026s
I20260924 00:21:07.566051 10625 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b8d74761b0b4fd0a22021d3870ec05c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:07 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:07.599380 11040 raft_consensus.cc:348] T 5e624be739394e6d87d6b3fef2f46302 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:07.599977 11040 raft_consensus.cc:374] T 5e624be739394e6d87d6b3fef2f46302 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:07.600198 11040 raft_consensus.cc:729] T 5e624be739394e6d87d6b3fef2f46302 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:07.600808 11040 consensus_queue.cc:260] T 5e624be739394e6d87d6b3fef2f46302 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
W20260924 00:21:07.602284 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet e937cee22326474598cef5c1d09b59c2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e937cee22326474598cef5c1d09b59c2 already in progress: creating tablet
I20260924 00:21:07.603206 11335 tablet_replica.cc:330] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:07.603111 11457 raft_consensus.cc:482] T 18abdec265a24dc8bfc765d4395767c0 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:07.603521 11457 raft_consensus.cc:504] T 18abdec265a24dc8bfc765d4395767c0 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:07.604574 11335 raft_consensus.cc:2229] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:07.605600 11335 raft_consensus.cc:2258] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:07.607093 11335 ts_tablet_manager.cc:1905] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:07.609510 11457 leader_election.cc:288] T 18abdec265a24dc8bfc765d4395767c0 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:07.614457 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18abdec265a24dc8bfc765d4395767c0" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:07.615477 10772 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e624be739394e6d87d6b3fef2f46302 with delete_type TABLET_DATA_DELETED (replaced by tablet 6bcfd729e9bd4040b3384ab24164e80d) from {username='slave'} at 127.0.0.1:35996
W20260924 00:21:07.616142 10592 leader_election.cc:341] T 18abdec265a24dc8bfc765d4395767c0 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:07.617498 11040 ts_tablet_manager.cc:1428] T 5e624be739394e6d87d6b3fef2f46302 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.021s user 0.005s sys 0.000s
I20260924 00:21:07.619620 10365 master_service.cc:946] ReplaceTablet: received request to replace tablet 772fb3d6180e499daf94ceeb9b26048d from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:07.628509 11040 tablet_bootstrap.cc:492] T bf144ab1a6464a4ab02ac0642c5255e3 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:07.632592 10374 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_0488528e57a740c0acd6a71051b64e6a" } modify_external_catalogs: true
I20260924 00:21:07.633248 11040 tablet_bootstrap.cc:654] T bf144ab1a6464a4ab02ac0642c5255e3 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:07.633122 10374 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_0488528e57a740c0acd6a71051b64e6a" } modify_external_catalogs: true
I20260924 00:21:07.639228 11335 ts_tablet_manager.cc:1918] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:07.639575 11335 log.cc:1196] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/c7e50ac8b7f947b1bfba8cd1cff0ea7e
I20260924 00:21:07.641873 11051 ts_tablet_manager.cc:1905] T 2964cd68d5e94f3780a6a3157bd86983 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:07.653115 11335 ts_tablet_manager.cc:1939] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:07.657829 10910 tablet_service.cc:1467] Processing CreateTablet for tablet 811a6a39194f4fb293bc61da1beacb62 (DEFAULT_TABLE table=default.Table_c41522845ef54f468c7686bc6b171fe6 [id=3f1f2d3ebded41a389cf2e4aca0e060e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.658454 11041 tablet_bootstrap.cc:654] T 60ebab3e93f54383b87aeb54bbde722d P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:07.659016 10910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 811a6a39194f4fb293bc61da1beacb62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.661682 10907 tablet_service.cc:1467] Processing CreateTablet for tablet df24a60e710241de9110afc5a6167bf9 (DEFAULT_TABLE table=default.Table_3f26bb58c6374373a24eb69faf5d68c6 [id=c5d285065f184765b488817af0b5d87d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.662830 10907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df24a60e710241de9110afc5a6167bf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.665822 10631 tablet_service.cc:1467] Processing CreateTablet for tablet 9e44244181324288ad04afb7096839cb (DEFAULT_TABLE table=default.Table_dae0ffc8534047588f6f7cb13a610af5 [id=138be013fbcb4f01a6358d062200fe8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.666986 10631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e44244181324288ad04afb7096839cb. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:21:07.677924 10361 consensus_peers.cc:487] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 -> Peer f8bd0b3184f84f5f8da10c9c21472f92 (127.10.4.124:35345): Couldn't send request to peer f8bd0b3184f84f5f8da10c9c21472f92. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.124:35345: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260924 00:21:07.678488 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18abdec265a24dc8bfc765d4395767c0" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
W20260924 00:21:07.681692 10589 leader_election.cc:341] T 18abdec265a24dc8bfc765d4395767c0 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:07.682133 10589 leader_election.cc:302] T 18abdec265a24dc8bfc765d4395767c0 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:07.683282 11457 raft_consensus.cc:2726] T 18abdec265a24dc8bfc765d4395767c0 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:07.648586 11342 raft_consensus.cc:504] T 5f2a32d5afff458fab79ffa106a98884 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:07.692679 11342 leader_election.cc:288] T 5f2a32d5afff458fab79ffa106a98884 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:07.693722 10656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f2a32d5afff458fab79ffa106a98884" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:07.694326 10656 raft_consensus.cc:2454] T 5f2a32d5afff458fab79ffa106a98884 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
I20260924 00:21:07.695329 11457 raft_consensus.cc:482] T 8a9443159a334093b4135364057e0c8b P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:07.695791 11457 raft_consensus.cc:504] T 8a9443159a334093b4135364057e0c8b P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:07.696625 10625 tablet_service.cc:1467] Processing CreateTablet for tablet a0668cc6cbc547c5bdc6b62a1b38fcde (DEFAULT_TABLE table=default.Table_153086de28a74809b8415724f702a78e [id=222d2010881c4b5c9635aa3c28adc72b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.697682 10625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0668cc6cbc547c5bdc6b62a1b38fcde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.698457 11457 leader_election.cc:288] T 8a9443159a334093b4135364057e0c8b P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:07.705624 10628 tablet_service.cc:1467] Processing CreateTablet for tablet 73bca39d6635475d83b38d11ce2a19d0 (DEFAULT_TABLE table=default.Table_d853fb40e0554242976678966283153e [id=3e6243f4f29c41db9fc70e43473ca92a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.713752 10628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73bca39d6635475d83b38d11ce2a19d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.694629 10915 tablet_service.cc:1467] Processing CreateTablet for tablet 9e44244181324288ad04afb7096839cb (DEFAULT_TABLE table=default.Table_dae0ffc8534047588f6f7cb13a610af5 [id=138be013fbcb4f01a6358d062200fe8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.715188 10915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e44244181324288ad04afb7096839cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.695590 10876 leader_election.cc:302] T 5f2a32d5afff458fab79ffa106a98884 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:07.729765 10906 tablet_service.cc:1467] Processing CreateTablet for tablet a0668cc6cbc547c5bdc6b62a1b38fcde (DEFAULT_TABLE table=default.Table_153086de28a74809b8415724f702a78e [id=222d2010881c4b5c9635aa3c28adc72b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.731058 10906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0668cc6cbc547c5bdc6b62a1b38fcde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.700299 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a9443159a334093b4135364057e0c8b" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:07.702663 10912 tablet_service.cc:1467] Processing CreateTablet for tablet 73bca39d6635475d83b38d11ce2a19d0 (DEFAULT_TABLE table=default.Table_d853fb40e0554242976678966283153e [id=3e6243f4f29c41db9fc70e43473ca92a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.734721 10912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73bca39d6635475d83b38d11ce2a19d0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:21:07.737326 10589 leader_election.cc:341] T 8a9443159a334093b4135364057e0c8b P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
W20260924 00:21:07.742697 10366 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet df03f899e5de4c12b16fbb40bdb1f33a consensus state has no leader
I20260924 00:21:07.745548 10366 master_service.cc:946] ReplaceTablet: received request to replace tablet 79d8dc7403f448df8cb960311eefe110 from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:07.747812 11342 raft_consensus.cc:2781] T 5f2a32d5afff458fab79ffa106a98884 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:07.748227 11342 raft_consensus.cc:482] T 5f2a32d5afff458fab79ffa106a98884 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:07.748587 11342 raft_consensus.cc:3037] T 5f2a32d5afff458fab79ffa106a98884 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:07.749150 10368 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_9dea9b14b2a04787987acb15887351a3" } modify_external_catalogs: true
I20260924 00:21:07.749646 10368 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_9dea9b14b2a04787987acb15887351a3" } modify_external_catalogs: true
W20260924 00:21:07.752691 10358 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 60ebab3e93f54383b87aeb54bbde722d (table default.Table_88117cdd98a74bcfa88b82c3162e7119 [id=ed6b4628a295498da00f778ad37db72e], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 60ebab3e93f54383b87aeb54bbde722d consensus state has no leader
I20260924 00:21:07.757988 10372 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_540e09a2364b4d07a38f2015f21fd31e" } new_table_name: "default.Table_82b79ff8ffca4a7880702b55cd10e393" modify_external_catalogs: true
I20260924 00:21:07.778555 11039 tablet_bootstrap.cc:492] T 4420cead1f284af8958f189557833098 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:07.779000 11039 ts_tablet_manager.cc:1397] T 4420cead1f284af8958f189557833098 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.265s user 0.013s sys 0.004s
I20260924 00:21:07.779379 10632 tablet_service.cc:1467] Processing CreateTablet for tablet a53491b1d40143caa73e802304436574 (DEFAULT_TABLE table=default.Table_2c873f940f8249f9950e78d583d64e36 [id=21a378fad88643b79609e92dc1b96bff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.779273 11342 raft_consensus.cc:504] T 5f2a32d5afff458fab79ffa106a98884 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:07.780390 10632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a53491b1d40143caa73e802304436574. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.781040 11039 raft_consensus.cc:348] T 4420cead1f284af8958f189557833098 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:07.781638 11039 raft_consensus.cc:374] T 4420cead1f284af8958f189557833098 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:07.781879 11039 raft_consensus.cc:729] T 4420cead1f284af8958f189557833098 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:07.782557 11342 leader_election.cc:288] T 5f2a32d5afff458fab79ffa106a98884 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:07.782696 11039 consensus_queue.cc:260] T 4420cead1f284af8958f189557833098 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:07.784421 11039 ts_tablet_manager.cc:1428] T 4420cead1f284af8958f189557833098 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260924 00:21:07.785971 10656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f2a32d5afff458fab79ffa106a98884" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:21:07.786495 10656 raft_consensus.cc:3037] T 5f2a32d5afff458fab79ffa106a98884 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:07.786788 11051 ts_tablet_manager.cc:1918] T 2964cd68d5e94f3780a6a3157bd86983 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:07.787094 11051 log.cc:1196] T 2964cd68d5e94f3780a6a3157bd86983 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/2964cd68d5e94f3780a6a3157bd86983
I20260924 00:21:07.787066 11457 raft_consensus.cc:482] T 4420cead1f284af8958f189557833098 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:07.787705 11051 ts_tablet_manager.cc:1939] T 2964cd68d5e94f3780a6a3157bd86983 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:07.787521 11457 raft_consensus.cc:504] T 4420cead1f284af8958f189557833098 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:07.789232 11457 leader_election.cc:288] T 4420cead1f284af8958f189557833098 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:07.794278 10656 raft_consensus.cc:2454] T 5f2a32d5afff458fab79ffa106a98884 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 1.
I20260924 00:21:07.795454 10876 leader_election.cc:302] T 5f2a32d5afff458fab79ffa106a98884 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:07.796844 11481 raft_consensus.cc:2781] T 5f2a32d5afff458fab79ffa106a98884 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:07.797263 11481 raft_consensus.cc:686] T 5f2a32d5afff458fab79ffa106a98884 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Becoming Leader. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Running, Role: LEADER
I20260924 00:21:07.798743 10909 tablet_service.cc:1467] Processing CreateTablet for tablet a53491b1d40143caa73e802304436574 (DEFAULT_TABLE table=default.Table_2c873f940f8249f9950e78d583d64e36 [id=21a378fad88643b79609e92dc1b96bff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.800987 11335 tablet_replica.cc:330] T 0d555f059f84478eb805f4ca4534dab4 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:07.801657 11335 raft_consensus.cc:2229] T 0d555f059f84478eb805f4ca4534dab4 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:07.802070 11335 raft_consensus.cc:2258] T 0d555f059f84478eb805f4ca4534dab4 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:07.802213 10634 tablet_service.cc:1467] Processing CreateTablet for tablet ac8eb7c81dcb4fa099a10d511882c0d4 (DEFAULT_TABLE table=default.Table_4c16876881e6429fbe20b9eb73c68d3f [id=a672b0959c50464eb890db93fd96c64c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.798502 11481 consensus_queue.cc:237] T 5f2a32d5afff458fab79ffa106a98884 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:07.803093 10634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac8eb7c81dcb4fa099a10d511882c0d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.803529 11335 ts_tablet_manager.cc:1905] T 0d555f059f84478eb805f4ca4534dab4 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:07.806762 10780 tablet_service.cc:1467] Processing CreateTablet for tablet 9e44244181324288ad04afb7096839cb (DEFAULT_TABLE table=default.Table_dae0ffc8534047588f6f7cb13a610af5 [id=138be013fbcb4f01a6358d062200fe8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.807634 10780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e44244181324288ad04afb7096839cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.811146 10799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a9443159a334093b4135364057e0c8b" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:07.811353 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet c7e50ac8b7f947b1bfba8cd1cff0ea7e (table default.Table_593b4d43f8144c04ab048e8221b3fc8e [id=951c867d04f74728a684d20d1104c054]) successfully deleted
I20260924 00:21:07.812281 10800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f2a32d5afff458fab79ffa106a98884" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:21:07.813378 10773 tablet_service.cc:1467] Processing CreateTablet for tablet a53491b1d40143caa73e802304436574 (DEFAULT_TABLE table=default.Table_2c873f940f8249f9950e78d583d64e36 [id=21a378fad88643b79609e92dc1b96bff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.813951 10777 tablet_service.cc:1467] Processing CreateTablet for tablet a0668cc6cbc547c5bdc6b62a1b38fcde (DEFAULT_TABLE table=default.Table_153086de28a74809b8415724f702a78e [id=222d2010881c4b5c9635aa3c28adc72b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.814229 10773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a53491b1d40143caa73e802304436574. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.814854 10777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0668cc6cbc547c5bdc6b62a1b38fcde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.815691 10774 tablet_service.cc:1467] Processing CreateTablet for tablet 73bca39d6635475d83b38d11ce2a19d0 (DEFAULT_TABLE table=default.Table_d853fb40e0554242976678966283153e [id=3e6243f4f29c41db9fc70e43473ca92a]), partition=RANGE (key) PARTITION UNBOUNDED
W20260924 00:21:07.815886 10592 leader_election.cc:341] T 8a9443159a334093b4135364057e0c8b P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:07.816247 10592 leader_election.cc:302] T 8a9443159a334093b4135364057e0c8b P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:07.816988 10775 tablet_service.cc:1467] Processing CreateTablet for tablet ac8eb7c81dcb4fa099a10d511882c0d4 (DEFAULT_TABLE table=default.Table_4c16876881e6429fbe20b9eb73c68d3f [id=a672b0959c50464eb890db93fd96c64c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.817229 11457 raft_consensus.cc:2726] T 8a9443159a334093b4135364057e0c8b P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:07.817641 11041 tablet_bootstrap.cc:492] T 60ebab3e93f54383b87aeb54bbde722d P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:07.818109 10775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac8eb7c81dcb4fa099a10d511882c0d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.818089 11041 ts_tablet_manager.cc:1397] T 60ebab3e93f54383b87aeb54bbde722d P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.254s user 0.014s sys 0.001s
I20260924 00:21:07.818645 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73bca39d6635475d83b38d11ce2a19d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.820603 11457 raft_consensus.cc:482] T a0f71049f6b3478da3b07ebc9293ce64 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:07.820566 11482 consensus_queue.cc:1035] T 119d83562ef546cab496cd7b44fecc15 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Connected to new peer: Peer: permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:07.820994 11457 raft_consensus.cc:504] T a0f71049f6b3478da3b07ebc9293ce64 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:07.822674 11457 leader_election.cc:288] T a0f71049f6b3478da3b07ebc9293ce64 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:07.823577 10939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0f71049f6b3478da3b07ebc9293ce64" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
W20260924 00:21:07.825100 10589 leader_election.cc:341] T a0f71049f6b3478da3b07ebc9293ce64 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:07.826962 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4420cead1f284af8958f189557833098" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:07.827497 10942 raft_consensus.cc:2454] T 4420cead1f284af8958f189557833098 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b10a9082a37b46f09d581944914c1670 in term 0.
I20260924 00:21:07.832058 10367 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 5f2a32d5afff458fab79ffa106a98884 (table default.Table_bf61c7623556425688b99c1c373e4879 [id=4f70e833d7f74d91af0faa01ed52f78b]). Expected version 1 got 0
I20260924 00:21:07.833313 10589 leader_election.cc:302] T 4420cead1f284af8958f189557833098 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:07.834158 11457 raft_consensus.cc:2781] T 4420cead1f284af8958f189557833098 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:07.834453 11457 raft_consensus.cc:482] T 4420cead1f284af8958f189557833098 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:07.834705 11457 raft_consensus.cc:3037] T 4420cead1f284af8958f189557833098 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:07.835741 11049 ts_tablet_manager.cc:1918] T e0074b1184ec4883ab1016330d98c968 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:21:07.836073 11049 log.cc:1196] T e0074b1184ec4883ab1016330d98c968 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/e0074b1184ec4883ab1016330d98c968
I20260924 00:21:07.836725 10920 tablet_service.cc:1514] Processing DeleteTablet for tablet f3432ac96da94d5099bcf9764c2b32e1 with delete_type TABLET_DATA_DELETED (replaced by tablet 5b43ce1fed0749b8b076b9b026914e07) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:07.843708 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4420cead1f284af8958f189557833098" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:07.844434 11359 raft_consensus.cc:482] T f3432ac96da94d5099bcf9764c2b32e1 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:07.842581 11457 raft_consensus.cc:504] T 4420cead1f284af8958f189557833098 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:07.844833 11359 raft_consensus.cc:504] T f3432ac96da94d5099bcf9764c2b32e1 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:07.846756 11359 leader_election.cc:288] T f3432ac96da94d5099bcf9764c2b32e1 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:07.812301 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f2a32d5afff458fab79ffa106a98884" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:07.849655 11457 leader_election.cc:288] T 4420cead1f284af8958f189557833098 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Requested vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
W20260924 00:21:07.853024 10592 leader_election.cc:341] T 4420cead1f284af8958f189557833098 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
W20260924 00:21:07.854682 10878 leader_election.cc:341] T 5f2a32d5afff458fab79ffa106a98884 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
W20260924 00:21:07.855460 10878 leader_election.cc:341] T 5f2a32d5afff458fab79ffa106a98884 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:07.855691 10365 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 772fb3d6180e499daf94ceeb9b26048d
I20260924 00:21:07.856971 10365 catalog_manager.cc:6565] ReplaceTablet: tablet 772fb3d6180e499daf94ceeb9b26048d deleted and replaced by tablet 0856ff2896ce4832af348947f226ee7b
I20260924 00:21:07.859010 10912 tablet_service.cc:1514] Processing DeleteTablet for tablet 772fb3d6180e499daf94ceeb9b26048d with delete_type TABLET_DATA_DELETED (replaced by tablet 0856ff2896ce4832af348947f226ee7b) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:07.859611 10625 tablet_service.cc:1514] Processing DeleteTablet for tablet 772fb3d6180e499daf94ceeb9b26048d with delete_type TABLET_DATA_DELETED (replaced by tablet 0856ff2896ce4832af348947f226ee7b) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:07.864112 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3432ac96da94d5099bcf9764c2b32e1" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:07.864672 10942 raft_consensus.cc:2454] T f3432ac96da94d5099bcf9764c2b32e1 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 0.
I20260924 00:21:07.865327 10939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4420cead1f284af8958f189557833098" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef"
I20260924 00:21:07.865821 10939 raft_consensus.cc:3037] T 4420cead1f284af8958f189557833098 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:07.866458 11342 raft_consensus.cc:482] T f3432ac96da94d5099bcf9764c2b32e1 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:07.866880 11342 raft_consensus.cc:504] T f3432ac96da94d5099bcf9764c2b32e1 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:07.867518 10778 tablet_service.cc:1467] Processing CreateTablet for tablet df03f899e5de4c12b16fbb40bdb1f33a (DEFAULT_TABLE table=default.Table_5db49282d679494c84606725e942b198 [id=64a8cee150dd4ad4bba25adf7dbe36e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.820323 11041 raft_consensus.cc:348] T 60ebab3e93f54383b87aeb54bbde722d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:07.872025 11041 raft_consensus.cc:374] T 60ebab3e93f54383b87aeb54bbde722d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:07.872277 11041 raft_consensus.cc:729] T 60ebab3e93f54383b87aeb54bbde722d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:07.872962 11041 consensus_queue.cc:260] T 60ebab3e93f54383b87aeb54bbde722d P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:07.874879 11342 leader_election.cc:288] T f3432ac96da94d5099bcf9764c2b32e1 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:07.878891 10365 catalog_manager.cc:5495] T 5f2a32d5afff458fab79ffa106a98884 P c8626e300d0644b0908797ad61bd7cef reported cstate change: term changed from 0 to 1, leader changed from <none> to c8626e300d0644b0908797ad61bd7cef (127.10.4.67). New cstate: current_term: 1 leader_uuid: "c8626e300d0644b0908797ad61bd7cef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } health_report { overall_health: UNKNOWN } } }
I20260924 00:21:07.880101 10365 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 5f2a32d5afff458fab79ffa106a98884 (table default.Table_bf61c7623556425688b99c1c373e4879 [id=4f70e833d7f74d91af0faa01ed52f78b]). Expected version 1 got 0
W20260924 00:21:07.883760 10876 leader_election.cc:341] T f3432ac96da94d5099bcf9764c2b32e1 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: f3432ac96da94d5099bcf9764c2b32e1
I20260924 00:21:07.884414 10625 tablet_service.cc:1467] Processing CreateTablet for tablet df03f899e5de4c12b16fbb40bdb1f33a (DEFAULT_TABLE table=default.Table_5db49282d679494c84606725e942b198 [id=64a8cee150dd4ad4bba25adf7dbe36e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.885484 10625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df03f899e5de4c12b16fbb40bdb1f33a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.885576 10939 raft_consensus.cc:2454] T 4420cead1f284af8958f189557833098 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b10a9082a37b46f09d581944914c1670 in term 1.
I20260924 00:21:07.891247 10912 tablet_service.cc:1467] Processing CreateTablet for tablet ac8eb7c81dcb4fa099a10d511882c0d4 (DEFAULT_TABLE table=default.Table_4c16876881e6429fbe20b9eb73c68d3f [id=a672b0959c50464eb890db93fd96c64c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.892221 10912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac8eb7c81dcb4fa099a10d511882c0d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.893625 10920 tablet_service.cc:1467] Processing CreateTablet for tablet df03f899e5de4c12b16fbb40bdb1f33a (DEFAULT_TABLE table=default.Table_5db49282d679494c84606725e942b198 [id=64a8cee150dd4ad4bba25adf7dbe36e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.894640 10920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df03f899e5de4c12b16fbb40bdb1f33a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.896983 11051 tablet_replica.cc:330] T 0d555f059f84478eb805f4ca4534dab4 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:07.897667 11051 raft_consensus.cc:2229] T 0d555f059f84478eb805f4ca4534dab4 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:07.898052 11051 raft_consensus.cc:2258] T 0d555f059f84478eb805f4ca4534dab4 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:07.899547 11051 ts_tablet_manager.cc:1905] T 0d555f059f84478eb805f4ca4534dab4 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:07.901399 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 2964cd68d5e94f3780a6a3157bd86983 (table default.Table_89e5d14641754be1934c7e3c880b21ee [id=fd5c26e6faab4fffb49cea70d6ab78a5]) successfully deleted
I20260924 00:21:07.902454 10909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a53491b1d40143caa73e802304436574. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.904063 11039 tablet_bootstrap.cc:492] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:07.907799 10779 tablet_service.cc:1467] Processing CreateTablet for tablet 83d67392af814e43a0fa1bee970820b8 (DEFAULT_TABLE table=default.Table_9e6a0e1a9c4f44fba041a24df1d601ac [id=02df80a9de5947dbbf8484189637255b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.912825 10589 leader_election.cc:302] T 4420cead1f284af8958f189557833098 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:07.913779 11457 raft_consensus.cc:2781] T 4420cead1f284af8958f189557833098 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:07.915015 11049 ts_tablet_manager.cc:1939] T e0074b1184ec4883ab1016330d98c968 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:07.915308 10367 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 60ebab3e93f54383b87aeb54bbde722d (table default.Table_88117cdd98a74bcfa88b82c3162e7119 [id=ed6b4628a295498da00f778ad37db72e]). Expected version 2 got 0
I20260924 00:21:07.919831 11039 tablet_bootstrap.cc:654] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:07.920241 10374 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet e46f92f2a020402681aa597e57f37c41
I20260924 00:21:07.919572 11457 raft_consensus.cc:686] T 4420cead1f284af8958f189557833098 P b10a9082a37b46f09d581944914c1670 [term 1 LEADER]: Becoming Leader. State: Replica: b10a9082a37b46f09d581944914c1670, State: Running, Role: LEADER
I20260924 00:21:07.922422 11457 consensus_queue.cc:237] T 4420cead1f284af8958f189557833098 P b10a9082a37b46f09d581944914c1670 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:07.923080 10915 tablet_service.cc:1514] Processing DeleteTablet for tablet e46f92f2a020402681aa597e57f37c41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:07 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:07.927959 11041 ts_tablet_manager.cc:1428] T 60ebab3e93f54383b87aeb54bbde722d P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.109s user 0.005s sys 0.000s
I20260924 00:21:07.928700 10631 tablet_service.cc:1514] Processing DeleteTablet for tablet e46f92f2a020402681aa597e57f37c41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:07 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:07.930995 10772 tablet_service.cc:1514] Processing DeleteTablet for tablet 772fb3d6180e499daf94ceeb9b26048d with delete_type TABLET_DATA_DELETED (replaced by tablet 0856ff2896ce4832af348947f226ee7b) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:07.932853 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4420cead1f284af8958f189557833098" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:21:07.935030 10779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83d67392af814e43a0fa1bee970820b8. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:21:07.935099 10592 leader_election.cc:341] T 4420cead1f284af8958f189557833098 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:07.935922 10778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df03f899e5de4c12b16fbb40bdb1f33a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.939991 10627 tablet_service.cc:1467] Processing CreateTablet for tablet 83d67392af814e43a0fa1bee970820b8 (DEFAULT_TABLE table=default.Table_9e6a0e1a9c4f44fba041a24df1d601ac [id=02df80a9de5947dbbf8484189637255b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:07.944499 10374 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_4541883eff524699a7226b51ad3b9727"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:07.945891 10374 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4541883eff524699a7226b51ad3b9727 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:07.946749 10369 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_3879c2e59faa453281d3a65092107295" } new_table_name: "default.Table_ed94e40b06bf4340bd3a775e16a6d139" modify_external_catalogs: true
I20260924 00:21:07.955915 10367 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_5e4d36ef35c84b2ba0dcf2eec1b33822"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:07.957583 10367 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5e4d36ef35c84b2ba0dcf2eec1b33822 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:07.870357 10799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0f71049f6b3478da3b07ebc9293ce64" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:07.889144 10733 leader_election.cc:341] T f3432ac96da94d5099bcf9764c2b32e1 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: f3432ac96da94d5099bcf9764c2b32e1
W20260924 00:21:07.961648 10592 leader_election.cc:341] T a0f71049f6b3478da3b07ebc9293ce64 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
W20260924 00:21:07.961688 10733 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:33461, user_credentials={real_user=slave}} blocked reactor thread for 79202.8us
I20260924 00:21:07.962015 10592 leader_election.cc:302] T a0f71049f6b3478da3b07ebc9293ce64 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:07.962636 11457 raft_consensus.cc:2726] T a0f71049f6b3478da3b07ebc9293ce64 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:07.963951 10732 leader_election.cc:302] T f3432ac96da94d5099bcf9764c2b32e1 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters: b10a9082a37b46f09d581944914c1670
I20260924 00:21:07.964648 11359 raft_consensus.cc:2781] T f3432ac96da94d5099bcf9764c2b32e1 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:07.965001 11359 raft_consensus.cc:482] T f3432ac96da94d5099bcf9764c2b32e1 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:07.965307 11359 raft_consensus.cc:3037] T f3432ac96da94d5099bcf9764c2b32e1 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:07.965675 10772 tablet_service.cc:1514] Processing DeleteTablet for tablet e46f92f2a020402681aa597e57f37c41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:07 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:07.971825 11359 raft_consensus.cc:504] T f3432ac96da94d5099bcf9764c2b32e1 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:07.974538 11359 leader_election.cc:288] T f3432ac96da94d5099bcf9764c2b32e1 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Requested vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:07.975203 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3432ac96da94d5099bcf9764c2b32e1" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:07.975989 10795 raft_consensus.cc:2379] T f3432ac96da94d5099bcf9764c2b32e1 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c8626e300d0644b0908797ad61bd7cef in current term 1: Already voted for candidate 48fbd603abe146c585d15196e1b5be08 in this term.
I20260924 00:21:07.979679 10370 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet 119d83562ef546cab496cd7b44fecc15 (table default.Table_76ffb857cf034b6f80598edd90dad953 [id=5fb5e1e90c9440f385e7b04520bcba9c]). Expected version 1 got 0
W20260924 00:21:07.982415 10733 leader_election.cc:341] T f3432ac96da94d5099bcf9764c2b32e1 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: f3432ac96da94d5099bcf9764c2b32e1
I20260924 00:21:07.983709 10627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83d67392af814e43a0fa1bee970820b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:07.983639 10371 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_f0b5a94e5ae7437383840e76682bd5fa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:07.983937 11482 consensus_queue.cc:1035] T 119d83562ef546cab496cd7b44fecc15 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Connected to new peer: Peer: permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260924 00:21:07.985054 10371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f0b5a94e5ae7437383840e76682bd5fa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260924 00:21:07.964775 10732 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:41301, user_credentials={real_user=slave}} blocked reactor thread for 82181.4us
I20260924 00:21:07.986502 10939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3432ac96da94d5099bcf9764c2b32e1" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef"
I20260924 00:21:07.986994 10939 raft_consensus.cc:3037] T f3432ac96da94d5099bcf9764c2b32e1 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:08.005517 11051 ts_tablet_manager.cc:1918] T 0d555f059f84478eb805f4ca4534dab4 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:08.005666 10939 raft_consensus.cc:2454] T f3432ac96da94d5099bcf9764c2b32e1 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 1.
I20260924 00:21:08.009553 11051 log.cc:1196] T 0d555f059f84478eb805f4ca4534dab4 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/0d555f059f84478eb805f4ca4534dab4
I20260924 00:21:08.011684 10878 leader_election.cc:302] T f3432ac96da94d5099bcf9764c2b32e1 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670
I20260924 00:21:08.013401 11342 raft_consensus.cc:2726] T f3432ac96da94d5099bcf9764c2b32e1 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:08.014487 10732 leader_election.cc:302] T f3432ac96da94d5099bcf9764c2b32e1 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters: b10a9082a37b46f09d581944914c1670
I20260924 00:21:08.015285 11457 raft_consensus.cc:482] T 60ebab3e93f54383b87aeb54bbde722d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:08.015700 11457 raft_consensus.cc:504] T 60ebab3e93f54383b87aeb54bbde722d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:08.017562 11457 leader_election.cc:288] T 60ebab3e93f54383b87aeb54bbde722d P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:07.983668 11041 tablet_bootstrap.cc:492] T a72742864a6b4299b71757fb308dcadf P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:08.018548 11051 ts_tablet_manager.cc:1939] T 0d555f059f84478eb805f4ca4534dab4 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:08.019992 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60ebab3e93f54383b87aeb54bbde722d" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:08.021327 10592 leader_election.cc:341] T 60ebab3e93f54383b87aeb54bbde722d P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:08.022240 10939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60ebab3e93f54383b87aeb54bbde722d" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
W20260924 00:21:08.023429 10589 leader_election.cc:341] T 60ebab3e93f54383b87aeb54bbde722d P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:08.023798 10589 leader_election.cc:302] T 60ebab3e93f54383b87aeb54bbde722d P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:08.024922 11457 raft_consensus.cc:2726] T 60ebab3e93f54383b87aeb54bbde722d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:08.025261 11041 tablet_bootstrap.cc:654] T a72742864a6b4299b71757fb308dcadf P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:08.025710 11049 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 209) took 1204 ms. Trace:
I20260924 00:21:08.028658 10373 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_b07b221a38e14a93aa27e7c2dfe55549"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:08.028755 11049 rpcz_store.cc:276] 0924 00:21:06.821003 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:06.825044 (+ 4041us) service_pool.cc:227] Handling call
0924 00:21:07.386962 (+561918us) ts_tablet_manager.cc:1145] Deleting tablet e0074b1184ec4883ab1016330d98c968
0924 00:21:07.386995 (+ 33us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:07.687748 (+300753us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:07.835519 (+147771us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:08.025645 (+190126us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":561345}
I20260924 00:21:08.029920 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet e0074b1184ec4883ab1016330d98c968 (table default.Table_63927aa1872e41da917d6b92c8acb310 [id=485c42c942b74abbb32bea36d25bc184]) successfully deleted
I20260924 00:21:08.029731 11359 raft_consensus.cc:2781] T f3432ac96da94d5099bcf9764c2b32e1 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader election won for term 1
W20260924 00:21:08.030146 10373 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b07b221a38e14a93aa27e7c2dfe55549 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:08.033727 11039 tablet_bootstrap.cc:492] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:08.034210 11039 ts_tablet_manager.cc:1397] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.130s user 0.010s sys 0.000s
I20260924 00:21:08.036273 11039 raft_consensus.cc:348] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:08.038499 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 0d555f059f84478eb805f4ca4534dab4 (table default.Table_99da18207fc542fbb68972724c4d410e [id=f75c1da3bd734e92bc105c473236fa9b]) successfully deleted
I20260924 00:21:08.038614 11049 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 208) took 1218 ms. Trace:
I20260924 00:21:08.038887 11049 rpcz_store.cc:276] 0924 00:21:06.820646 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:06.837063 (+ 16417us) service_pool.cc:227] Handling call
0924 00:21:08.038296 (+1201233us) ts_tablet_manager.cc:1145] Deleting tablet a72742864a6b4299b71757fb308dcadf
0924 00:21:08.038339 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:08.038565 (+ 226us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1200568}
W20260924 00:21:08.040886 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet a72742864a6b4299b71757fb308dcadf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a72742864a6b4299b71757fb308dcadf already in progress: creating tablet
I20260924 00:21:08.042088 11039 raft_consensus.cc:374] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:08.042343 11039 raft_consensus.cc:729] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
W20260924 00:21:08.043147 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 2499754460e04c27ac53d52a4a67f557 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2499754460e04c27ac53d52a4a67f557 already in progress: creating tablet
W20260924 00:21:08.044286 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet cb51d5c6460049108cd7e1974821e578 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cb51d5c6460049108cd7e1974821e578 already in progress: creating tablet
I20260924 00:21:08.044744 11051 tablet_replica.cc:330] T b06bf1e4691441769e162aa5404d4656 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:08.045473 11051 raft_consensus.cc:2229] T b06bf1e4691441769e162aa5404d4656 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:08.045871 11051 raft_consensus.cc:2258] T b06bf1e4691441769e162aa5404d4656 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:08.047350 11051 ts_tablet_manager.cc:1905] T b06bf1e4691441769e162aa5404d4656 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:08.047896 11049 tablet_replica.cc:330] T 8a9443159a334093b4135364057e0c8b P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:08.049292 10920 tablet_service.cc:1467] Processing CreateTablet for tablet 83d67392af814e43a0fa1bee970820b8 (DEFAULT_TABLE table=default.Table_9e6a0e1a9c4f44fba041a24df1d601ac [id=02df80a9de5947dbbf8484189637255b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:08.049480 11049 raft_consensus.cc:2229] T 8a9443159a334093b4135364057e0c8b P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:08.049857 11049 raft_consensus.cc:2258] T 8a9443159a334093b4135364057e0c8b P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:08.050390 10920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83d67392af814e43a0fa1bee970820b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:08.051030 11560 raft_consensus.cc:482] T 5e624be739394e6d87d6b3fef2f46302 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:08.051366 11049 ts_tablet_manager.cc:1905] T 8a9443159a334093b4135364057e0c8b P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:08.043541 11039 consensus_queue.cc:260] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:08.052901 11560 raft_consensus.cc:504] T 5e624be739394e6d87d6b3fef2f46302 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:08.066461 10366 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 79d8dc7403f448df8cb960311eefe110
W20260924 00:21:08.068683 10372 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4aa0dc5c502a406b99609f044b6baeb0 has no consensus state
I20260924 00:21:08.069188 11560 leader_election.cc:288] T 5e624be739394e6d87d6b3fef2f46302 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:08.069433 10366 catalog_manager.cc:6565] ReplaceTablet: tablet 79d8dc7403f448df8cb960311eefe110 deleted and replaced by tablet ba31383ba3a04edab4c250c4b3146185
I20260924 00:21:08.070319 11039 ts_tablet_manager.cc:1428] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.036s user 0.005s sys 0.000s
I20260924 00:21:08.073177 10368 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 0410cf6bd23f4e30ba480087efae21c5
I20260924 00:21:08.074553 10939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e624be739394e6d87d6b3fef2f46302" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:08.075119 10939 raft_consensus.cc:2454] T 5e624be739394e6d87d6b3fef2f46302 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 0.
I20260924 00:21:08.077776 10627 tablet_service.cc:1514] Processing DeleteTablet for tablet 79d8dc7403f448df8cb960311eefe110 with delete_type TABLET_DATA_DELETED (replaced by tablet ba31383ba3a04edab4c250c4b3146185) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:08.080549 10912 tablet_service.cc:1514] Processing DeleteTablet for tablet 79d8dc7403f448df8cb960311eefe110 with delete_type TABLET_DATA_DELETED (replaced by tablet ba31383ba3a04edab4c250c4b3146185) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:08.081507 11039 tablet_bootstrap.cc:492] T 18abdec265a24dc8bfc765d4395767c0 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
W20260924 00:21:08.072234 10733 leader_election.cc:341] T 5e624be739394e6d87d6b3fef2f46302 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: 5e624be739394e6d87d6b3fef2f46302
I20260924 00:21:08.072897 10780 tablet_service.cc:1514] Processing DeleteTablet for tablet 79d8dc7403f448df8cb960311eefe110 with delete_type TABLET_DATA_DELETED (replaced by tablet ba31383ba3a04edab4c250c4b3146185) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:08.082403 11335 ts_tablet_manager.cc:1918] T 0d555f059f84478eb805f4ca4534dab4 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:08.085681 11335 log.cc:1196] T 0d555f059f84478eb805f4ca4534dab4 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/0d555f059f84478eb805f4ca4534dab4
I20260924 00:21:08.086138 10732 leader_election.cc:302] T 5e624be739394e6d87d6b3fef2f46302 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters: b10a9082a37b46f09d581944914c1670
I20260924 00:21:08.087253 11560 raft_consensus.cc:2781] T 5e624be739394e6d87d6b3fef2f46302 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:08.088238 11560 raft_consensus.cc:482] T 5e624be739394e6d87d6b3fef2f46302 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:08.088985 11560 raft_consensus.cc:3037] T 5e624be739394e6d87d6b3fef2f46302 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:08.099396 11049 ts_tablet_manager.cc:1918] T 8a9443159a334093b4135364057e0c8b P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:08.086319 11335 ts_tablet_manager.cc:1939] T 0d555f059f84478eb805f4ca4534dab4 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:08.102144 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 0d555f059f84478eb805f4ca4534dab4 (table default.Table_99da18207fc542fbb68972724c4d410e [id=f75c1da3bd734e92bc105c473236fa9b]) successfully deleted
I20260924 00:21:08.103039 11335 tablet_replica.cc:330] T f3432ac96da94d5099bcf9764c2b32e1 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:08.104828 11049 log.cc:1196] T 8a9443159a334093b4135364057e0c8b P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/8a9443159a334093b4135364057e0c8b
I20260924 00:21:08.105991 11049 ts_tablet_manager.cc:1939] T 8a9443159a334093b4135364057e0c8b P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:08.106019 11359 raft_consensus.cc:686] T f3432ac96da94d5099bcf9764c2b32e1 P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: Becoming Leader. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Running, Role: LEADER
I20260924 00:21:08.107391 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 8a9443159a334093b4135364057e0c8b (table default.Table_0179ec39d29c4bad9ed0def94394be47 [id=68d73a1a9f1540809f974b9a21f44c1b]) successfully deleted
I20260924 00:21:08.107632 11049 tablet_replica.cc:330] T 4420cead1f284af8958f189557833098 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:08.107234 11359 consensus_queue.cc:237] T f3432ac96da94d5099bcf9764c2b32e1 P 48fbd603abe146c585d15196e1b5be08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:08.108492 11049 raft_consensus.cc:2229] T 4420cead1f284af8958f189557833098 P b10a9082a37b46f09d581944914c1670 [term 1 LEADER]: Raft consensus shutting down.
W20260924 00:21:08.108675 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet f85ac5b0fd2348d88e1076b781424ab3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f85ac5b0fd2348d88e1076b781424ab3 already in progress: creating tablet
I20260924 00:21:08.114256 11049 pending_rounds.cc:62] T 4420cead1f284af8958f189557833098 P b10a9082a37b46f09d581944914c1670: Trying to abort 1 pending ops.
I20260924 00:21:08.114683 11039 tablet_bootstrap.cc:654] T 18abdec265a24dc8bfc765d4395767c0 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:08.114606 11049 pending_rounds.cc:68] T 4420cead1f284af8958f189557833098 P b10a9082a37b46f09d581944914c1670: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332697161418768384 op_type: NO_OP noop_request { }
I20260924 00:21:08.115105 11049 raft_consensus.cc:2866] T 4420cead1f284af8958f189557833098 P b10a9082a37b46f09d581944914c1670 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260924 00:21:08.115474 11049 raft_consensus.cc:2258] T 4420cead1f284af8958f189557833098 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:08.120198 11040 tablet_bootstrap.cc:492] T bf144ab1a6464a4ab02ac0642c5255e3 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:08.120579 11040 ts_tablet_manager.cc:1397] T bf144ab1a6464a4ab02ac0642c5255e3 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.492s user 0.007s sys 0.002s
I20260924 00:21:08.120699 11049 ts_tablet_manager.cc:1905] T 4420cead1f284af8958f189557833098 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:08.118788 11560 raft_consensus.cc:504] T 5e624be739394e6d87d6b3fef2f46302 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:08.122969 11040 raft_consensus.cc:348] T bf144ab1a6464a4ab02ac0642c5255e3 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:08.123672 11040 raft_consensus.cc:374] T bf144ab1a6464a4ab02ac0642c5255e3 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:08.123926 11040 raft_consensus.cc:729] T bf144ab1a6464a4ab02ac0642c5255e3 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:08.124234 10368 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_88117cdd98a74bcfa88b82c3162e7119" } modify_external_catalogs: true
I20260924 00:21:08.124802 10368 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_88117cdd98a74bcfa88b82c3162e7119" } modify_external_catalogs: true
I20260924 00:21:08.129339 10939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e624be739394e6d87d6b3fef2f46302" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef"
I20260924 00:21:08.129874 10939 raft_consensus.cc:3037] T 5e624be739394e6d87d6b3fef2f46302 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
W20260924 00:21:08.130364 10733 leader_election.cc:341] T 5e624be739394e6d87d6b3fef2f46302 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: 5e624be739394e6d87d6b3fef2f46302
I20260924 00:21:08.126417 11039 tablet_bootstrap.cc:492] T 18abdec265a24dc8bfc765d4395767c0 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:08.131709 11039 ts_tablet_manager.cc:1397] T 18abdec265a24dc8bfc765d4395767c0 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.050s user 0.004s sys 0.006s
I20260924 00:21:08.134146 11560 leader_election.cc:288] T 5e624be739394e6d87d6b3fef2f46302 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Requested vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:08.138450 10939 raft_consensus.cc:2454] T 5e624be739394e6d87d6b3fef2f46302 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 1.
I20260924 00:21:08.139798 10732 leader_election.cc:302] T 5e624be739394e6d87d6b3fef2f46302 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters: b10a9082a37b46f09d581944914c1670
I20260924 00:21:08.140089 11039 raft_consensus.cc:348] T 18abdec265a24dc8bfc765d4395767c0 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:08.140717 11039 raft_consensus.cc:374] T 18abdec265a24dc8bfc765d4395767c0 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:08.143628 11039 raft_consensus.cc:729] T 18abdec265a24dc8bfc765d4395767c0 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:08.144203 11039 consensus_queue.cc:260] T 18abdec265a24dc8bfc765d4395767c0 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:08.157289 11359 raft_consensus.cc:2781] T 5e624be739394e6d87d6b3fef2f46302 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:08.157369 11335 raft_consensus.cc:2229] T f3432ac96da94d5099bcf9764c2b32e1 P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:21:08.157799 11359 raft_consensus.cc:686] T 5e624be739394e6d87d6b3fef2f46302 P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: Becoming Leader. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Running, Role: LEADER
I20260924 00:21:08.158521 11359 consensus_queue.cc:237] T 5e624be739394e6d87d6b3fef2f46302 P 48fbd603abe146c585d15196e1b5be08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:08.166013 11335 pending_rounds.cc:62] T f3432ac96da94d5099bcf9764c2b32e1 P 48fbd603abe146c585d15196e1b5be08: Trying to abort 1 pending ops.
I20260924 00:21:08.166280 11335 pending_rounds.cc:68] T f3432ac96da94d5099bcf9764c2b32e1 P 48fbd603abe146c585d15196e1b5be08: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332697162175700992 op_type: NO_OP noop_request { }
I20260924 00:21:08.166627 11335 raft_consensus.cc:2866] T f3432ac96da94d5099bcf9764c2b32e1 P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260924 00:21:08.166661 11051 ts_tablet_manager.cc:1918] T b06bf1e4691441769e162aa5404d4656 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:08.166917 11335 raft_consensus.cc:2258] T f3432ac96da94d5099bcf9764c2b32e1 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:08.167027 11051 log.cc:1196] T b06bf1e4691441769e162aa5404d4656 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/b06bf1e4691441769e162aa5404d4656
I20260924 00:21:08.167613 11051 ts_tablet_manager.cc:1939] T b06bf1e4691441769e162aa5404d4656 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:08.168541 11041 tablet_bootstrap.cc:492] T a72742864a6b4299b71757fb308dcadf P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:08.169042 11041 ts_tablet_manager.cc:1397] T a72742864a6b4299b71757fb308dcadf P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.186s user 0.010s sys 0.000s
I20260924 00:21:08.170050 11335 ts_tablet_manager.cc:1905] T f3432ac96da94d5099bcf9764c2b32e1 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:08.173800 11039 ts_tablet_manager.cc:1428] T 18abdec265a24dc8bfc765d4395767c0 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.042s user 0.003s sys 0.002s
I20260924 00:21:08.174613 11039 tablet_bootstrap.cc:492] T 4325b60efc1649c9ba4828d0a098a676 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:08.188007 11039 tablet_bootstrap.cc:654] T 4325b60efc1649c9ba4828d0a098a676 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:08.191691 11049 ts_tablet_manager.cc:1918] T 4420cead1f284af8958f189557833098 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:08.192020 11049 log.cc:1196] T 4420cead1f284af8958f189557833098 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/4420cead1f284af8958f189557833098
I20260924 00:21:08.193125 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet b06bf1e4691441769e162aa5404d4656 (table default.Table_6a2c714f26254ca89f7021633fc89fb7 [id=534f5d0c2e6e426e834efb9b089e4651]) successfully deleted
W20260924 00:21:08.193809 10358 catalog_manager.cc:4808] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 0980afc2af174da2a8ca2dd414e9825c because the tablet was not found. No further retry: Not found: Tablet not found: 0980afc2af174da2a8ca2dd414e9825c
I20260924 00:21:08.194006 11051 tablet_replica.cc:330] T ffa011cf8dc446de849bea75a4f20056 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:08.194684 11051 raft_consensus.cc:2229] T ffa011cf8dc446de849bea75a4f20056 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:08.195080 11051 raft_consensus.cc:2258] T ffa011cf8dc446de849bea75a4f20056 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:08.201915 11051 ts_tablet_manager.cc:1905] T ffa011cf8dc446de849bea75a4f20056 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:08.207139 10909 tablet_service.cc:1514] Processing DeleteTablet for tablet 4420cead1f284af8958f189557833098 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:01 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:08.214879 10365 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet 119d83562ef546cab496cd7b44fecc15 (table default.Table_76ffb857cf034b6f80598edd90dad953 [id=5fb5e1e90c9440f385e7b04520bcba9c]). Expected version 1 got 0
I20260924 00:21:08.217973 11049 ts_tablet_manager.cc:1939] T 4420cead1f284af8958f189557833098 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:08.218801 11039 tablet_bootstrap.cc:492] T 4325b60efc1649c9ba4828d0a098a676 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:08.218861 11049 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 213) took 1021 ms. Trace:
I20260924 00:21:08.221133 11049 rpcz_store.cc:276] 0924 00:21:07.197820 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:07.199635 (+ 1815us) service_pool.cc:227] Handling call
0924 00:21:08.107503 (+907868us) ts_tablet_manager.cc:1145] Deleting tablet 4420cead1f284af8958f189557833098
0924 00:21:08.107537 (+ 34us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:08.148700 (+ 41163us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:08.191499 (+ 42799us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:08.218814 (+ 27315us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":906835}
I20260924 00:21:08.222440 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 4420cead1f284af8958f189557833098 (table default.Table_bdbcdcde9ac241d5b22b5e8b0da6a9a7 [id=21e1b3346ba346c883fced1e7a1b6cf6]) successfully deleted
I20260924 00:21:08.219174 11039 ts_tablet_manager.cc:1397] T 4325b60efc1649c9ba4828d0a098a676 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.045s user 0.007s sys 0.000s
I20260924 00:21:08.224803 11041 raft_consensus.cc:348] T a72742864a6b4299b71757fb308dcadf P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:08.225453 11041 raft_consensus.cc:374] T a72742864a6b4299b71757fb308dcadf P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:08.225697 11041 raft_consensus.cc:729] T a72742864a6b4299b71757fb308dcadf P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:08.226305 11049 tablet_replica.cc:330] T 0c583d9ba1414f8ea2e605b4b6b076f8 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:08.227105 11071 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.124:35345
W20260924 00:21:08.229687 10369 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 63e9ebe18f824e18999c6aa5d8286f83 consensus state has no leader
I20260924 00:21:08.231518 11071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/master-2/data/info.pb
I20260924 00:21:08.166502 11040 consensus_queue.cc:260] T bf144ab1a6464a4ab02ac0642c5255e3 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:08.234474 11049 raft_consensus.cc:2229] T 0c583d9ba1414f8ea2e605b4b6b076f8 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:08.234424 11039 raft_consensus.cc:348] T 4325b60efc1649c9ba4828d0a098a676 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:08.234921 11049 raft_consensus.cc:2258] T 0c583d9ba1414f8ea2e605b4b6b076f8 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:08.235023 11039 raft_consensus.cc:374] T 4325b60efc1649c9ba4828d0a098a676 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:08.235260 11039 raft_consensus.cc:729] T 4325b60efc1649c9ba4828d0a098a676 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:08.226614 11572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.4.124:35345 every 8 connection(s)
I20260924 00:21:08.236457 11049 ts_tablet_manager.cc:1905] T 0c583d9ba1414f8ea2e605b4b6b076f8 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:08.236632 10257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu as pid 11071
I20260924 00:21:08.244540 11335 ts_tablet_manager.cc:1918] T f3432ac96da94d5099bcf9764c2b32e1 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:08.244913 11335 log.cc:1196] T f3432ac96da94d5099bcf9764c2b32e1 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/f3432ac96da94d5099bcf9764c2b32e1
I20260924 00:21:08.245615 11335 ts_tablet_manager.cc:1939] T f3432ac96da94d5099bcf9764c2b32e1 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:08.245234 11039 consensus_queue.cc:260] T 4325b60efc1649c9ba4828d0a098a676 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:08.247852 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet f3432ac96da94d5099bcf9764c2b32e1 (table default.Table_9608bad42525495eaa4df537b37969e7 [id=aa74d52589984aec8e6d296e5dffd623]) successfully deleted
I20260924 00:21:08.249854 11040 ts_tablet_manager.cc:1428] T bf144ab1a6464a4ab02ac0642c5255e3 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.129s user 0.005s sys 0.001s
I20260924 00:21:08.250749 11040 tablet_bootstrap.cc:492] T 5f2a32d5afff458fab79ffa106a98884 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:08.226390 11041 consensus_queue.cc:260] T a72742864a6b4299b71757fb308dcadf P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:08.264217 11041 ts_tablet_manager.cc:1428] T a72742864a6b4299b71757fb308dcadf P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.095s user 0.005s sys 0.000s
I20260924 00:21:08.267187 10365 master_service.cc:946] ReplaceTablet: received request to replace tablet 915e8072851c4d2b90f8af12b1dc58e0 from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:08.273295 11041 tablet_bootstrap.cc:492] T 2499754460e04c27ac53d52a4a67f557 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:08.285365 11039 ts_tablet_manager.cc:1428] T 4325b60efc1649c9ba4828d0a098a676 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.060s user 0.004s sys 0.001s
W20260924 00:21:08.297425 10361 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.10.4.66:35359, user_credentials={real_user=slave}} blocked reactor thread for 49608.3us
I20260924 00:21:08.302031 11039 tablet_bootstrap.cc:492] T bf144ab1a6464a4ab02ac0642c5255e3 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:08.302552 10372 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_ff0bbc1b70c2417e9c69a7f7492b2981" } new_table_name: "default.Table_e412cbac77574c5aafc587eaa698b633" modify_external_catalogs: true
I20260924 00:21:08.304708 10369 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet bf144ab1a6464a4ab02ac0642c5255e3 (table default.Table_382099cf77a24bb8bd6fe084926431c9 [id=3db896d304514ead8dfe726c402a6fcd]). Expected version 1 got 0
I20260924 00:21:08.312634 11051 ts_tablet_manager.cc:1918] T ffa011cf8dc446de849bea75a4f20056 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:08.316162 10369 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet bf144ab1a6464a4ab02ac0642c5255e3 (table default.Table_382099cf77a24bb8bd6fe084926431c9 [id=3db896d304514ead8dfe726c402a6fcd]). Expected version 1 got 0
I20260924 00:21:08.316352 11039 tablet_bootstrap.cc:654] T bf144ab1a6464a4ab02ac0642c5255e3 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:08.319022 10369 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_4a5edca73a094b4d93ba4d389679ab48" } modify_external_catalogs: true
I20260924 00:21:08.324169 11040 tablet_bootstrap.cc:654] T 5f2a32d5afff458fab79ffa106a98884 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:08.325066 11051 log.cc:1196] T ffa011cf8dc446de849bea75a4f20056 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/ffa011cf8dc446de849bea75a4f20056
W20260924 00:21:08.332898 10876 consensus_peers.cc:487] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P c8626e300d0644b0908797ad61bd7cef -> Peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Couldn't send request to peer b10a9082a37b46f09d581944914c1670. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c7e50ac8b7f947b1bfba8cd1cff0ea7e. This is attempt 1: this message will repeat every 5th retry.
I20260924 00:21:08.325851 11051 ts_tablet_manager.cc:1939] T ffa011cf8dc446de849bea75a4f20056 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:08.335840 11051 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35358 (request call id 214) took 1100 ms. Trace:
I20260924 00:21:08.336181 11051 rpcz_store.cc:276] 0924 00:21:07.235524 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:07.235679 (+ 155us) service_pool.cc:227] Handling call
0924 00:21:08.193870 (+958191us) ts_tablet_manager.cc:1145] Deleting tablet ffa011cf8dc446de849bea75a4f20056
0924 00:21:08.193904 (+ 34us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:08.244586 (+ 50682us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:08.312428 (+ 67842us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:08.335783 (+ 23355us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":957685}
I20260924 00:21:08.337560 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet ffa011cf8dc446de849bea75a4f20056 (table default.Table_6cf69012192f4f1087e29db70ae8c282 [id=93c1734de58245f8b1d5ab88ec9f0c8e]) successfully deleted
I20260924 00:21:08.339250 11573 sys_catalog.cc:263] Verifying existing consensus state
I20260924 00:21:08.319749 10369 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_4a5edca73a094b4d93ba4d389679ab48" } modify_external_catalogs: true
I20260924 00:21:08.343146 11051 tablet_replica.cc:330] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
W20260924 00:21:08.340349 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet e937cee22326474598cef5c1d09b59c2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e937cee22326474598cef5c1d09b59c2 already in progress: creating tablet
I20260924 00:21:08.343833 11051 raft_consensus.cc:2229] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Raft consensus shutting down.
W20260924 00:21:08.343911 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 3b8d74761b0b4fd0a22021d3870ec05c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3b8d74761b0b4fd0a22021d3870ec05c already in progress: creating tablet
I20260924 00:21:08.344535 11051 pending_rounds.cc:62] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P c8626e300d0644b0908797ad61bd7cef: Trying to abort 1 pending ops.
I20260924 00:21:08.346575 11039 tablet_bootstrap.cc:492] T bf144ab1a6464a4ab02ac0642c5255e3 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:08.346962 11039 ts_tablet_manager.cc:1397] T bf144ab1a6464a4ab02ac0642c5255e3 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.045s user 0.005s sys 0.003s
I20260924 00:21:08.349335 10845 heartbeater.cc:347] Connected to a master server at 127.10.4.124:35345
I20260924 00:21:08.349078 11039 raft_consensus.cc:348] T bf144ab1a6464a4ab02ac0642c5255e3 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:08.349700 11039 raft_consensus.cc:374] T bf144ab1a6464a4ab02ac0642c5255e3 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:08.349979 11039 raft_consensus.cc:729] T bf144ab1a6464a4ab02ac0642c5255e3 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:08.344743 11051 pending_rounds.cc:68] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P c8626e300d0644b0908797ad61bd7cef: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332697159840546816 op_type: NO_OP noop_request { }
I20260924 00:21:08.363092 11051 raft_consensus.cc:2866] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260924 00:21:08.363458 11051 raft_consensus.cc:2258] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:08.368435 10701 heartbeater.cc:347] Connected to a master server at 127.10.4.124:35345
I20260924 00:21:08.369246 11041 tablet_bootstrap.cc:654] T 2499754460e04c27ac53d52a4a67f557 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:08.376441 10988 heartbeater.cc:347] Connected to a master server at 127.10.4.124:35345
I20260924 00:21:08.378401 11039 consensus_queue.cc:260] T bf144ab1a6464a4ab02ac0642c5255e3 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:08.380254 11039 ts_tablet_manager.cc:1428] T bf144ab1a6464a4ab02ac0642c5255e3 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.033s user 0.005s sys 0.000s
I20260924 00:21:08.383404 11051 ts_tablet_manager.cc:1905] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:08.389734 11039 tablet_bootstrap.cc:492] T 8a9443159a334093b4135364057e0c8b P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:08.405948 10988 heartbeater.cc:347] Connected to a master server at 127.10.4.124:35345
I20260924 00:21:08.417004 10367 master_service.cc:946] ReplaceTablet: received request to replace tablet 84e73180fab74524b4d568a010dc7175 from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:08.417104 11335 tablet_replica.cc:330] T 5e624be739394e6d87d6b3fef2f46302 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:08.417840 11335 raft_consensus.cc:2229] T 5e624be739394e6d87d6b3fef2f46302 P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:21:08.418577 11335 pending_rounds.cc:62] T 5e624be739394e6d87d6b3fef2f46302 P 48fbd603abe146c585d15196e1b5be08: Trying to abort 1 pending ops.
I20260924 00:21:08.418834 11335 pending_rounds.cc:68] T 5e624be739394e6d87d6b3fef2f46302 P 48fbd603abe146c585d15196e1b5be08: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332697162399932416 op_type: NO_OP noop_request { }
I20260924 00:21:08.419157 11335 raft_consensus.cc:2866] T 5e624be739394e6d87d6b3fef2f46302 P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260924 00:21:08.419440 11335 raft_consensus.cc:2258] T 5e624be739394e6d87d6b3fef2f46302 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 00:21:08.420374 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 3b8d74761b0b4fd0a22021d3870ec05c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3b8d74761b0b4fd0a22021d3870ec05c already in progress: creating tablet
I20260924 00:21:08.420598 11039 tablet_bootstrap.cc:654] T 8a9443159a334093b4135364057e0c8b P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:08.422015 11573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92: Bootstrap starting.
I20260924 00:21:08.422705 11335 ts_tablet_manager.cc:1905] T 5e624be739394e6d87d6b3fef2f46302 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:08.431447 11049 ts_tablet_manager.cc:1918] T 0c583d9ba1414f8ea2e605b4b6b076f8 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:08.433529 10374 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_67493ad72aac4bbca26a6a322a60e5cd" } new_table_name: "default.Table_07390b277c404c7c9daecd17fe3dccca" modify_external_catalogs: true
I20260924 00:21:08.439584 11049 log.cc:1196] T 0c583d9ba1414f8ea2e605b4b6b076f8 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/0c583d9ba1414f8ea2e605b4b6b076f8
I20260924 00:21:08.441392 11049 ts_tablet_manager.cc:1939] T 0c583d9ba1414f8ea2e605b4b6b076f8 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:08.443579 11040 tablet_bootstrap.cc:492] T 5f2a32d5afff458fab79ffa106a98884 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:08.443958 11040 ts_tablet_manager.cc:1397] T 5f2a32d5afff458fab79ffa106a98884 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.193s user 0.008s sys 0.006s
I20260924 00:21:08.431880 11041 tablet_bootstrap.cc:492] T 2499754460e04c27ac53d52a4a67f557 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:08.448796 11041 ts_tablet_manager.cc:1397] T 2499754460e04c27ac53d52a4a67f557 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.176s user 0.010s sys 0.000s
I20260924 00:21:08.451305 11041 raft_consensus.cc:348] T 2499754460e04c27ac53d52a4a67f557 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:08.451938 11041 raft_consensus.cc:374] T 2499754460e04c27ac53d52a4a67f557 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:08.452174 11041 raft_consensus.cc:729] T 2499754460e04c27ac53d52a4a67f557 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:08.453356 11359 raft_consensus.cc:482] T bf144ab1a6464a4ab02ac0642c5255e3 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:08.453773 11359 raft_consensus.cc:504] T bf144ab1a6464a4ab02ac0642c5255e3 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:08.455610 11359 leader_election.cc:288] T bf144ab1a6464a4ab02ac0642c5255e3 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:08.456988 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf144ab1a6464a4ab02ac0642c5255e3" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:08.457540 10654 raft_consensus.cc:2454] T bf144ab1a6464a4ab02ac0642c5255e3 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 0.
I20260924 00:21:08.459100 10939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf144ab1a6464a4ab02ac0642c5255e3" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:08.459278 10733 leader_election.cc:302] T bf144ab1a6464a4ab02ac0642c5255e3 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670; no voters:
I20260924 00:21:08.459681 10939 raft_consensus.cc:2454] T bf144ab1a6464a4ab02ac0642c5255e3 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 0.
I20260924 00:21:08.459996 11359 raft_consensus.cc:2781] T bf144ab1a6464a4ab02ac0642c5255e3 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:08.460289 11359 raft_consensus.cc:482] T bf144ab1a6464a4ab02ac0642c5255e3 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:08.460563 11359 raft_consensus.cc:3037] T bf144ab1a6464a4ab02ac0642c5255e3 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:08.463410 11051 ts_tablet_manager.cc:1918] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:08.463752 11051 log.cc:1196] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/c7e50ac8b7f947b1bfba8cd1cff0ea7e
I20260924 00:21:08.464175 11049 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 218) took 1182 ms. Trace:
I20260924 00:21:08.464365 11051 ts_tablet_manager.cc:1939] T c7e50ac8b7f947b1bfba8cd1cff0ea7e P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:08.468647 11359 raft_consensus.cc:504] T bf144ab1a6464a4ab02ac0642c5255e3 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:08.469036 11049 rpcz_store.cc:276] 0924 00:21:07.281846 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:07.281978 (+ 132us) service_pool.cc:227] Handling call
0924 00:21:08.226154 (+944176us) ts_tablet_manager.cc:1145] Deleting tablet 0c583d9ba1414f8ea2e605b4b6b076f8
0924 00:21:08.226206 (+ 52us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:08.385723 (+159517us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:08.431248 (+ 45525us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:08.464119 (+ 32871us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":943133}
I20260924 00:21:08.470768 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet c7e50ac8b7f947b1bfba8cd1cff0ea7e (table default.Table_593b4d43f8144c04ab048e8221b3fc8e [id=951c867d04f74728a684d20d1104c054]) successfully deleted
I20260924 00:21:08.472386 11039 tablet_bootstrap.cc:492] T 8a9443159a334093b4135364057e0c8b P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:08.472741 11039 ts_tablet_manager.cc:1397] T 8a9443159a334093b4135364057e0c8b P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.083s user 0.009s sys 0.001s
I20260924 00:21:08.474818 11039 raft_consensus.cc:348] T 8a9443159a334093b4135364057e0c8b P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:08.475432 11039 raft_consensus.cc:374] T 8a9443159a334093b4135364057e0c8b P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:08.475665 11039 raft_consensus.cc:729] T 8a9443159a334093b4135364057e0c8b P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:08.476277 11039 consensus_queue.cc:260] T 8a9443159a334093b4135364057e0c8b P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:08.477321 11051 tablet_replica.cc:330] T 5e624be739394e6d87d6b3fef2f46302 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:08.478458 10939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf144ab1a6464a4ab02ac0642c5255e3" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef"
I20260924 00:21:08.478956 10939 raft_consensus.cc:3037] T bf144ab1a6464a4ab02ac0642c5255e3 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:08.479023 11051 raft_consensus.cc:2229] T 5e624be739394e6d87d6b3fef2f46302 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:08.479445 11051 raft_consensus.cc:2258] T 5e624be739394e6d87d6b3fef2f46302 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:08.480907 11051 ts_tablet_manager.cc:1905] T 5e624be739394e6d87d6b3fef2f46302 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:08.482347 11359 leader_election.cc:288] T bf144ab1a6464a4ab02ac0642c5255e3 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Requested vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:08.488902 10939 raft_consensus.cc:2454] T bf144ab1a6464a4ab02ac0642c5255e3 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 1.
I20260924 00:21:08.490193 10732 leader_election.cc:302] T bf144ab1a6464a4ab02ac0642c5255e3 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:08.490793 11039 ts_tablet_manager.cc:1428] T 8a9443159a334093b4135364057e0c8b P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.018s user 0.004s sys 0.000s
I20260924 00:21:08.491565 11039 tablet_bootstrap.cc:492] T a72742864a6b4299b71757fb308dcadf P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:08.492450 11573 log.cc:824] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92: Log is configured to *not* fsync() on all Append() calls
I20260924 00:21:08.492877 11359 raft_consensus.cc:2781] T bf144ab1a6464a4ab02ac0642c5255e3 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:08.493309 11359 raft_consensus.cc:686] T bf144ab1a6464a4ab02ac0642c5255e3 P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: Becoming Leader. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Running, Role: LEADER
I20260924 00:21:08.496538 11039 tablet_bootstrap.cc:654] T a72742864a6b4299b71757fb308dcadf P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:08.493997 11359 consensus_queue.cc:237] T bf144ab1a6464a4ab02ac0642c5255e3 P 48fbd603abe146c585d15196e1b5be08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:08.477869 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf144ab1a6464a4ab02ac0642c5255e3" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:21:08.497265 10654 raft_consensus.cc:3037] T bf144ab1a6464a4ab02ac0642c5255e3 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:08.504232 10654 raft_consensus.cc:2454] T bf144ab1a6464a4ab02ac0642c5255e3 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 1.
I20260924 00:21:08.506553 11049 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 221) took 1076 ms. Trace:
I20260924 00:21:08.506855 11049 rpcz_store.cc:276] 0924 00:21:07.430647 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:07.430760 (+ 113us) service_pool.cc:227] Handling call
0924 00:21:08.506248 (+1075488us) ts_tablet_manager.cc:1145] Deleting tablet e937cee22326474598cef5c1d09b59c2
0924 00:21:08.506288 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:08.506508 (+ 220us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1074971}
I20260924 00:21:08.508141 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 0c583d9ba1414f8ea2e605b4b6b076f8 (table default.Table_fd43d6016c81497586729ff4cbe3213c [id=70e3fa6998b649468ec8575758708401]) successfully deleted
I20260924 00:21:08.509109 11049 tablet_replica.cc:330] T a0f71049f6b3478da3b07ebc9293ce64 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
W20260924 00:21:08.510023 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet e937cee22326474598cef5c1d09b59c2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e937cee22326474598cef5c1d09b59c2 already in progress: creating tablet
I20260924 00:21:08.513173 11049 raft_consensus.cc:2229] T a0f71049f6b3478da3b07ebc9293ce64 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:08.514460 11049 raft_consensus.cc:2258] T a0f71049f6b3478da3b07ebc9293ce64 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:08.516065 11049 ts_tablet_manager.cc:1905] T a0f71049f6b3478da3b07ebc9293ce64 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:08.519626 10367 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 84e73180fab74524b4d568a010dc7175
I20260924 00:21:08.520823 10367 catalog_manager.cc:6565] ReplaceTablet: tablet 84e73180fab74524b4d568a010dc7175 deleted and replaced by tablet 1acc16e2d3e14ab092da083b720cc29a
I20260924 00:21:08.522269 10774 tablet_service.cc:1514] Processing DeleteTablet for tablet 84e73180fab74524b4d568a010dc7175 with delete_type TABLET_DATA_DELETED (replaced by tablet 1acc16e2d3e14ab092da083b720cc29a) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:08.523360 10920 tablet_service.cc:1514] Processing DeleteTablet for tablet 84e73180fab74524b4d568a010dc7175 with delete_type TABLET_DATA_DELETED (replaced by tablet 1acc16e2d3e14ab092da083b720cc29a) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:08.525038 10625 tablet_service.cc:1514] Processing DeleteTablet for tablet 84e73180fab74524b4d568a010dc7175 with delete_type TABLET_DATA_DELETED (replaced by tablet 1acc16e2d3e14ab092da083b720cc29a) from {username='slave'} at 127.0.0.1:46896
W20260924 00:21:08.527297 10372 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cf0f0f016d074b1fb78d9b427c67fe86 consensus state has no leader
I20260924 00:21:08.527655 10365 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 915e8072851c4d2b90f8af12b1dc58e0
I20260924 00:21:08.528766 10365 catalog_manager.cc:6565] ReplaceTablet: tablet 915e8072851c4d2b90f8af12b1dc58e0 deleted and replaced by tablet 8e99dc66ce9e41ef98b6b24fa41c1b36
I20260924 00:21:08.528714 11335 ts_tablet_manager.cc:1918] T 5e624be739394e6d87d6b3fef2f46302 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:08.529044 11335 log.cc:1196] T 5e624be739394e6d87d6b3fef2f46302 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/5e624be739394e6d87d6b3fef2f46302
I20260924 00:21:08.529675 11335 ts_tablet_manager.cc:1939] T 5e624be739394e6d87d6b3fef2f46302 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:08.531527 10774 tablet_service.cc:1514] Processing DeleteTablet for tablet 915e8072851c4d2b90f8af12b1dc58e0 with delete_type TABLET_DATA_DELETED (replaced by tablet 8e99dc66ce9e41ef98b6b24fa41c1b36) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:08.531606 10920 tablet_service.cc:1514] Processing DeleteTablet for tablet 915e8072851c4d2b90f8af12b1dc58e0 with delete_type TABLET_DATA_DELETED (replaced by tablet 8e99dc66ce9e41ef98b6b24fa41c1b36) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:08.533102 11051 ts_tablet_manager.cc:1918] T 5e624be739394e6d87d6b3fef2f46302 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:08.533227 10625 tablet_service.cc:1514] Processing DeleteTablet for tablet 915e8072851c4d2b90f8af12b1dc58e0 with delete_type TABLET_DATA_DELETED (replaced by tablet 8e99dc66ce9e41ef98b6b24fa41c1b36) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:08.533428 11051 log.cc:1196] T 5e624be739394e6d87d6b3fef2f46302 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/5e624be739394e6d87d6b3fef2f46302
I20260924 00:21:08.533981 11051 ts_tablet_manager.cc:1939] T 5e624be739394e6d87d6b3fef2f46302 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:08.535980 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 5e624be739394e6d87d6b3fef2f46302 (table default.Table_9670783a92074b639ecb0d6c374811e5 [id=df2a5a74db65431c9ecb750184f64709]) successfully deleted
I20260924 00:21:08.538499 11342 raft_consensus.cc:482] T 8a9443159a334093b4135364057e0c8b P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:08.538910 11342 raft_consensus.cc:504] T 8a9443159a334093b4135364057e0c8b P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:08.540655 11342 leader_election.cc:288] T 8a9443159a334093b4135364057e0c8b P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:08.541625 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a9443159a334093b4135364057e0c8b" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:08.542938 10878 leader_election.cc:341] T 8a9443159a334093b4135364057e0c8b P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:08.548784 11342 raft_consensus.cc:482] T 18abdec265a24dc8bfc765d4395767c0 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:08.466233 11040 raft_consensus.cc:348] T 5f2a32d5afff458fab79ffa106a98884 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:08.549356 11040 raft_consensus.cc:374] T 5f2a32d5afff458fab79ffa106a98884 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:08.549216 11342 raft_consensus.cc:504] T 18abdec265a24dc8bfc765d4395767c0 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:08.549609 11040 raft_consensus.cc:729] T 5f2a32d5afff458fab79ffa106a98884 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:08.551487 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18abdec265a24dc8bfc765d4395767c0" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:08.552609 10878 leader_election.cc:341] T 18abdec265a24dc8bfc765d4395767c0 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:08.553395 11342 leader_election.cc:288] T 18abdec265a24dc8bfc765d4395767c0 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:08.561331 11051 tablet_replica.cc:330] T f3432ac96da94d5099bcf9764c2b32e1 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:08.562034 11051 raft_consensus.cc:2229] T f3432ac96da94d5099bcf9764c2b32e1 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:08.562295 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 5e624be739394e6d87d6b3fef2f46302 (table default.Table_9670783a92074b639ecb0d6c374811e5 [id=df2a5a74db65431c9ecb750184f64709]) successfully deleted
I20260924 00:21:08.562444 11051 raft_consensus.cc:2258] T f3432ac96da94d5099bcf9764c2b32e1 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:08.564469 11041 consensus_queue.cc:260] T 2499754460e04c27ac53d52a4a67f557 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:08.569597 11335 tablet_replica.cc:330] T 772fb3d6180e499daf94ceeb9b26048d P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:08.570281 11335 raft_consensus.cc:2229] T 772fb3d6180e499daf94ceeb9b26048d P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:08.570681 11335 raft_consensus.cc:2258] T 772fb3d6180e499daf94ceeb9b26048d P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:08.565971 11040 consensus_queue.cc:260] T 5f2a32d5afff458fab79ffa106a98884 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:08.573906 11040 ts_tablet_manager.cc:1428] T 5f2a32d5afff458fab79ffa106a98884 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.130s user 0.004s sys 0.002s
I20260924 00:21:08.573689 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18abdec265a24dc8bfc765d4395767c0" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:08.574240 10654 raft_consensus.cc:2454] T 18abdec265a24dc8bfc765d4395767c0 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
I20260924 00:21:08.575435 11051 ts_tablet_manager.cc:1905] T f3432ac96da94d5099bcf9764c2b32e1 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:08.583554 11040 tablet_bootstrap.cc:492] T 18abdec265a24dc8bfc765d4395767c0 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:08.587795 11335 ts_tablet_manager.cc:1905] T 772fb3d6180e499daf94ceeb9b26048d P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:08.592970 10365 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_0a7f75ce29d74b7da36e39b072bffccc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:08.593683 10372 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_cd15e5e8e07947459840c3ed5dca1f1d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:08.596248 10372 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cd15e5e8e07947459840c3ed5dca1f1d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260924 00:21:08.596334 10365 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0a7f75ce29d74b7da36e39b072bffccc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:08.603753 11573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 00:21:08.604997 11573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92: Bootstrap complete.
I20260924 00:21:08.607404 11040 tablet_bootstrap.cc:654] T 18abdec265a24dc8bfc765d4395767c0 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:08.618686 10799 raft_consensus.cc:3037] T 5f2a32d5afff458fab79ffa106a98884 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
W20260924 00:21:08.618626 10876 leader_election.cc:341] T 8a9443159a334093b4135364057e0c8b P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: 8a9443159a334093b4135364057e0c8b
I20260924 00:21:08.619056 10876 leader_election.cc:302] T 8a9443159a334093b4135364057e0c8b P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670
I20260924 00:21:08.619716 11342 raft_consensus.cc:2726] T 8a9443159a334093b4135364057e0c8b P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:08.626302 11342 consensus_queue.cc:1035] T 5f2a32d5afff458fab79ffa106a98884 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Connected to new peer: Peer: permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:08.631358 11051 ts_tablet_manager.cc:1918] T f3432ac96da94d5099bcf9764c2b32e1 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:08.631711 11051 log.cc:1196] T f3432ac96da94d5099bcf9764c2b32e1 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/f3432ac96da94d5099bcf9764c2b32e1
I20260924 00:21:08.632292 11051 ts_tablet_manager.cc:1939] T f3432ac96da94d5099bcf9764c2b32e1 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:08.633436 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet f3432ac96da94d5099bcf9764c2b32e1 (table default.Table_9608bad42525495eaa4df537b37969e7 [id=aa74d52589984aec8e6d296e5dffd623]) successfully deleted
I20260924 00:21:08.620645 10876 leader_election.cc:302] T 18abdec265a24dc8bfc765d4395767c0 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08
I20260924 00:21:08.641834 11342 raft_consensus.cc:2781] T 18abdec265a24dc8bfc765d4395767c0 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:08.642220 11051 tablet_replica.cc:330] T 772fb3d6180e499daf94ceeb9b26048d P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:08.642162 11342 raft_consensus.cc:482] T 18abdec265a24dc8bfc765d4395767c0 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:08.642526 11342 raft_consensus.cc:3037] T 18abdec265a24dc8bfc765d4395767c0 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:08.648962 11342 raft_consensus.cc:504] T 18abdec265a24dc8bfc765d4395767c0 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:08.650717 11039 tablet_bootstrap.cc:492] T a72742864a6b4299b71757fb308dcadf P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:08.651100 11039 ts_tablet_manager.cc:1397] T a72742864a6b4299b71757fb308dcadf P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.160s user 0.007s sys 0.003s
I20260924 00:21:08.651741 11342 leader_election.cc:288] T 18abdec265a24dc8bfc765d4395767c0 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:08.652889 11051 raft_consensus.cc:2229] T 772fb3d6180e499daf94ceeb9b26048d P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:21:08.653789 11051 raft_consensus.cc:2258] T 772fb3d6180e499daf94ceeb9b26048d P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:08.654862 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18abdec265a24dc8bfc765d4395767c0" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:21:08.655380 10654 raft_consensus.cc:3037] T 18abdec265a24dc8bfc765d4395767c0 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:08.655671 11041 ts_tablet_manager.cc:1428] T 2499754460e04c27ac53d52a4a67f557 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.207s user 0.010s sys 0.000s
I20260924 00:21:08.657473 11041 tablet_bootstrap.cc:492] T 72d3bb80e1034881b94eb75de5d900c2 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:08.657680 11335 ts_tablet_manager.cc:1918] T 772fb3d6180e499daf94ceeb9b26048d P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:08.658025 11335 log.cc:1196] T 772fb3d6180e499daf94ceeb9b26048d P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/772fb3d6180e499daf94ceeb9b26048d
I20260924 00:21:08.658651 11335 ts_tablet_manager.cc:1939] T 772fb3d6180e499daf94ceeb9b26048d P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:08.661870 10654 raft_consensus.cc:2454] T 18abdec265a24dc8bfc765d4395767c0 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 1.
I20260924 00:21:08.662528 11041 tablet_bootstrap.cc:654] T 72d3bb80e1034881b94eb75de5d900c2 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:08.665050 10876 leader_election.cc:302] T 18abdec265a24dc8bfc765d4395767c0 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:08.666314 11342 raft_consensus.cc:2781] T 18abdec265a24dc8bfc765d4395767c0 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:08.666793 11342 raft_consensus.cc:686] T 18abdec265a24dc8bfc765d4395767c0 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Becoming Leader. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Running, Role: LEADER
I20260924 00:21:08.667467 11342 consensus_queue.cc:237] T 18abdec265a24dc8bfc765d4395767c0 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:08.669086 11049 ts_tablet_manager.cc:1918] T a0f71049f6b3478da3b07ebc9293ce64 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:08.679697 11049 log.cc:1196] T a0f71049f6b3478da3b07ebc9293ce64 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/a0f71049f6b3478da3b07ebc9293ce64
I20260924 00:21:08.681223 11049 ts_tablet_manager.cc:1939] T a0f71049f6b3478da3b07ebc9293ce64 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:08.682406 11051 ts_tablet_manager.cc:1905] T 772fb3d6180e499daf94ceeb9b26048d P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:08.683841 11593 raft_consensus.cc:482] T 4325b60efc1649c9ba4828d0a098a676 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:08.684317 11593 raft_consensus.cc:504] T 4325b60efc1649c9ba4828d0a098a676 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:08.686414 11049 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 227) took 1104 ms. Trace:
I20260924 00:21:08.687055 11593 leader_election.cc:288] T 4325b60efc1649c9ba4828d0a098a676 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:21:08.686995 10365 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_5d4b70d8740f435bb43cb05808ec557e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:08.688369 10365 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5d4b70d8740f435bb43cb05808ec557e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:08.689409 10372 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_aac740991490482b9f371d9cb2076e76"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:08.690721 10372 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aac740991490482b9f371d9cb2076e76 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:08.715404 11602 tablet.cc:1722] T 85a0ad61b7cb4e11b30e84ddc4655b64 P c8626e300d0644b0908797ad61bd7cef: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260924 00:21:08.693970 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4325b60efc1649c9ba4828d0a098a676" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:08.719223 10654 raft_consensus.cc:2454] T 4325b60efc1649c9ba4828d0a098a676 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
I20260924 00:21:08.720121 10876 leader_election.cc:302] T 4325b60efc1649c9ba4828d0a098a676 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:08.720146 11049 rpcz_store.cc:276] 0924 00:21:07.582001 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:07.582107 (+ 106us) service_pool.cc:227] Handling call
0924 00:21:08.508980 (+926873us) ts_tablet_manager.cc:1145] Deleting tablet a0f71049f6b3478da3b07ebc9293ce64
0924 00:21:08.509015 (+ 35us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:08.557767 (+ 48752us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:08.668864 (+111097us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:08.686360 (+ 17496us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":926352}
I20260924 00:21:08.720851 11593 raft_consensus.cc:2781] T 4325b60efc1649c9ba4828d0a098a676 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:08.721167 11593 raft_consensus.cc:482] T 4325b60efc1649c9ba4828d0a098a676 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:08.721453 11593 raft_consensus.cc:3037] T 4325b60efc1649c9ba4828d0a098a676 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:08.723079 11049 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 224) took 1186 ms. Trace:
I20260924 00:21:08.723361 11049 rpcz_store.cc:276] 0924 00:21:07.536256 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:07.565923 (+ 29667us) service_pool.cc:227] Handling call
0924 00:21:08.722757 (+1156834us) ts_tablet_manager.cc:1145] Deleting tablet 3b8d74761b0b4fd0a22021d3870ec05c
0924 00:21:08.722798 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:08.723032 (+ 234us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1126045}
I20260924 00:21:08.728695 11593 raft_consensus.cc:504] T 4325b60efc1649c9ba4828d0a098a676 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:08.745970 10367 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_9e6a0e1a9c4f44fba041a24df1d601ac" } new_table_name: "default.Table_b4a6ce0bb6394e0aadb54d15db1f4b80" modify_external_catalogs: true
I20260924 00:21:08.744982 10365 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_82ad1ad5194e43008bb85e5b99ae0d44" } new_table_name: "default.Table_60e366f2e65b432da448892cea15e440" modify_external_catalogs: true
I20260924 00:21:08.752615 11593 leader_election.cc:288] T 4325b60efc1649c9ba4828d0a098a676 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:21:08.752569 11604 tablet.cc:1722] T 85a0ad61b7cb4e11b30e84ddc4655b64 P b10a9082a37b46f09d581944914c1670: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260924 00:21:08.755695 11051 ts_tablet_manager.cc:1918] T 772fb3d6180e499daf94ceeb9b26048d P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:08.756124 11051 log.cc:1196] T 772fb3d6180e499daf94ceeb9b26048d P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/772fb3d6180e499daf94ceeb9b26048d
I20260924 00:21:08.756086 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4325b60efc1649c9ba4828d0a098a676" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:21:08.756644 10654 raft_consensus.cc:3037] T 4325b60efc1649c9ba4828d0a098a676 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:08.757316 11051 ts_tablet_manager.cc:1939] T 772fb3d6180e499daf94ceeb9b26048d P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:08.762758 10654 raft_consensus.cc:2454] T 4325b60efc1649c9ba4828d0a098a676 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 1.
I20260924 00:21:08.764344 11049 tablet_replica.cc:330] T 772fb3d6180e499daf94ceeb9b26048d P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:08.763720 10876 leader_election.cc:302] T 4325b60efc1649c9ba4828d0a098a676 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:08.765076 11049 raft_consensus.cc:2229] T 772fb3d6180e499daf94ceeb9b26048d P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:08.765233 11593 raft_consensus.cc:2781] T 4325b60efc1649c9ba4828d0a098a676 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:08.767774 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 772fb3d6180e499daf94ceeb9b26048d (table default.Table_9c2aa717e62340c785be379c0dd5837c [id=737634d9f0f4413c8b6dcaa26de87e2d]) successfully deleted
I20260924 00:21:08.768640 11049 raft_consensus.cc:2258] T 772fb3d6180e499daf94ceeb9b26048d P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:08.721849 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet a0f71049f6b3478da3b07ebc9293ce64 (table default.Table_9f2df24361da4f9a9340f6f8673043ea [id=59ec3e94e2f44150994a5ffc1d9fa8f4]) successfully deleted
W20260924 00:21:08.769296 10359 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.10.4.65:33461, user_credentials={real_user=slave}} blocked reactor thread for 47466.4us
W20260924 00:21:08.770673 10359 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet cf0f0f016d074b1fb78d9b427c67fe86 (table default.Table_e412cbac77574c5aafc587eaa698b633 [id=667f4349af3f4eb28461e33f65beeb1c], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet cf0f0f016d074b1fb78d9b427c67fe86 consensus state has no leader
W20260924 00:21:08.771095 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 3b8d74761b0b4fd0a22021d3870ec05c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3b8d74761b0b4fd0a22021d3870ec05c already in progress: creating tablet
I20260924 00:21:08.774837 11049 ts_tablet_manager.cc:1905] T 772fb3d6180e499daf94ceeb9b26048d P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260924 00:21:08.777598 10367 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 83d67392af814e43a0fa1bee970820b8 consensus state has no leader
I20260924 00:21:08.779595 11041 tablet_bootstrap.cc:492] T 72d3bb80e1034881b94eb75de5d900c2 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:08.785415 11041 ts_tablet_manager.cc:1397] T 72d3bb80e1034881b94eb75de5d900c2 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.128s user 0.005s sys 0.005s
I20260924 00:21:08.787694 11041 raft_consensus.cc:348] T 72d3bb80e1034881b94eb75de5d900c2 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:08.788297 11041 raft_consensus.cc:374] T 72d3bb80e1034881b94eb75de5d900c2 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:08.788537 11041 raft_consensus.cc:729] T 72d3bb80e1034881b94eb75de5d900c2 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:08.792577 11039 raft_consensus.cc:348] T a72742864a6b4299b71757fb308dcadf P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
W20260924 00:21:08.808740 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet e46f92f2a020402681aa597e57f37c41 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e46f92f2a020402681aa597e57f37c41 already in progress: creating tablet
I20260924 00:21:08.801209 11041 consensus_queue.cc:260] T 72d3bb80e1034881b94eb75de5d900c2 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:08.810539 11457 raft_consensus.cc:482] T a72742864a6b4299b71757fb308dcadf P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:08.810925 11457 raft_consensus.cc:504] T a72742864a6b4299b71757fb308dcadf P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:08.813165 11457 leader_election.cc:288] T a72742864a6b4299b71757fb308dcadf P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
W20260924 00:21:08.822376 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 79d8dc7403f448df8cb960311eefe110 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 79d8dc7403f448df8cb960311eefe110 already in progress: creating tablet
I20260924 00:21:08.828456 11041 ts_tablet_manager.cc:1428] T 72d3bb80e1034881b94eb75de5d900c2 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.043s user 0.006s sys 0.001s
I20260924 00:21:08.830156 11573 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } }
I20260924 00:21:08.833061 10367 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_69005ec88288442691befac5eff2adc7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:08.826733 11051 tablet_replica.cc:330] T 4420cead1f284af8958f189557833098 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:08.837998 11051 raft_consensus.cc:2229] T 4420cead1f284af8958f189557833098 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:08.838411 11051 raft_consensus.cc:2258] T 4420cead1f284af8958f189557833098 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 00:21:08.838879 10367 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_69005ec88288442691befac5eff2adc7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260924 00:21:08.839440 11599 log.cc:924] Time spent T 5f2a32d5afff458fab79ffa106a98884 P 48fbd603abe146c585d15196e1b5be08: Append to log took a long time: real 0.198s user 0.001s sys 0.000s
I20260924 00:21:08.840286 11051 ts_tablet_manager.cc:1905] T 4420cead1f284af8958f189557833098 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:08.841357 10800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4325b60efc1649c9ba4828d0a098a676" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:21:08.834400 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18abdec265a24dc8bfc765d4395767c0" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:21:08.843070 10798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4325b60efc1649c9ba4828d0a098a676" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:08.844663 10794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a72742864a6b4299b71757fb308dcadf" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:08.845119 11039 raft_consensus.cc:374] T a72742864a6b4299b71757fb308dcadf P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:08.845378 11039 raft_consensus.cc:729] T a72742864a6b4299b71757fb308dcadf P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
W20260924 00:21:08.845930 10878 leader_election.cc:341] T 4325b60efc1649c9ba4828d0a098a676 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:08.846091 11039 consensus_queue.cc:260] T a72742864a6b4299b71757fb308dcadf P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
W20260924 00:21:08.846704 10878 leader_election.cc:341] T 18abdec265a24dc8bfc765d4395767c0 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
W20260924 00:21:08.847647 10878 leader_election.cc:341] T 4325b60efc1649c9ba4828d0a098a676 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:08.848214 11039 ts_tablet_manager.cc:1428] T a72742864a6b4299b71757fb308dcadf P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.197s user 0.005s sys 0.001s
I20260924 00:21:08.829293 11593 raft_consensus.cc:686] T 4325b60efc1649c9ba4828d0a098a676 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Becoming Leader. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Running, Role: LEADER
I20260924 00:21:08.849705 11593 consensus_queue.cc:237] T 4325b60efc1649c9ba4828d0a098a676 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:08.844105 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 772fb3d6180e499daf94ceeb9b26048d (table default.Table_9c2aa717e62340c785be379c0dd5837c [id=737634d9f0f4413c8b6dcaa26de87e2d]) successfully deleted
W20260924 00:21:08.850937 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet e46f92f2a020402681aa597e57f37c41 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e46f92f2a020402681aa597e57f37c41 already in progress: creating tablet
W20260924 00:21:08.851498 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 79d8dc7403f448df8cb960311eefe110 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 79d8dc7403f448df8cb960311eefe110 already in progress: creating tablet
W20260924 00:21:08.852017 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 84e73180fab74524b4d568a010dc7175 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 84e73180fab74524b4d568a010dc7175 already in progress: creating tablet
W20260924 00:21:08.852538 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 915e8072851c4d2b90f8af12b1dc58e0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 915e8072851c4d2b90f8af12b1dc58e0 already in progress: creating tablet
I20260924 00:21:08.866104 11039 tablet_bootstrap.cc:492] T 60ebab3e93f54383b87aeb54bbde722d P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:08.833746 11573 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8bd0b3184f84f5f8da10c9c21472f92, State: Initialized, Role: FOLLOWER
I20260924 00:21:08.879313 11573 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } }
I20260924 00:21:08.881767 10372 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_81b3a0208acd453c8a35957f47f25ff3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:08.884555 11611 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } } }
I20260924 00:21:08.885360 11611 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:21:08.889999 11573 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [sys.catalog]: configured and running, proceeding with master startup.
W20260924 00:21:08.894342 10372 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_81b3a0208acd453c8a35957f47f25ff3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:08.897002 10373 master_service.cc:946] ReplaceTablet: received request to replace tablet 18abdec265a24dc8bfc765d4395767c0 from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:08.905105 11039 tablet_bootstrap.cc:654] T 60ebab3e93f54383b87aeb54bbde722d P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:08.913517 11051 ts_tablet_manager.cc:1918] T 4420cead1f284af8958f189557833098 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:08.914237 11593 consensus_queue.cc:1035] T 5f2a32d5afff458fab79ffa106a98884 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Connected to new peer: Peer: permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:08.915582 11039 tablet_bootstrap.cc:492] T 60ebab3e93f54383b87aeb54bbde722d P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:08.915953 11039 ts_tablet_manager.cc:1397] T 60ebab3e93f54383b87aeb54bbde722d P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.067s user 0.003s sys 0.005s
I20260924 00:21:08.913861 11051 log.cc:1196] T 4420cead1f284af8958f189557833098 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/4420cead1f284af8958f189557833098
I20260924 00:21:08.921440 10371 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_a214c5e4b9584a1fba04c74e906fe4d3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:08.922864 10371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a214c5e4b9584a1fba04c74e906fe4d3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:08.918146 11039 raft_consensus.cc:348] T 60ebab3e93f54383b87aeb54bbde722d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:08.925554 11039 raft_consensus.cc:374] T 60ebab3e93f54383b87aeb54bbde722d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:08.926062 11051 ts_tablet_manager.cc:1939] T 4420cead1f284af8958f189557833098 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:08.925793 11039 raft_consensus.cc:729] T 60ebab3e93f54383b87aeb54bbde722d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:08.929347 11039 consensus_queue.cc:260] T 60ebab3e93f54383b87aeb54bbde722d P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:08.931130 11039 ts_tablet_manager.cc:1428] T 60ebab3e93f54383b87aeb54bbde722d P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.015s user 0.006s sys 0.000s
I20260924 00:21:08.931864 11039 tablet_bootstrap.cc:492] T 0c583d9ba1414f8ea2e605b4b6b076f8 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
W20260924 00:21:08.836025 10589 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:41301, user_credentials={real_user=slave}} blocked reactor thread for 113573us
I20260924 00:21:08.935797 11041 tablet_bootstrap.cc:492] T b36cbb6520de4df58b88cfa53c20ee7a P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:08.836660 11605 raft_consensus.cc:482] T 2499754460e04c27ac53d52a4a67f557 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:08.936182 11605 raft_consensus.cc:504] T 2499754460e04c27ac53d52a4a67f557 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:08.938058 11605 leader_election.cc:288] T 2499754460e04c27ac53d52a4a67f557 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:08.945806 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a72742864a6b4299b71757fb308dcadf" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:08.946382 10942 raft_consensus.cc:2454] T a72742864a6b4299b71757fb308dcadf P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b10a9082a37b46f09d581944914c1670 in term 0.
W20260924 00:21:08.853667 10592 leader_election.cc:341] T a72742864a6b4299b71757fb308dcadf P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
W20260924 00:21:08.948804 10592 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:35359, user_credentials={real_user=slave}} blocked reactor thread for 95164.5us
I20260924 00:21:08.883632 11606 raft_consensus.cc:482] T 72d3bb80e1034881b94eb75de5d900c2 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:08.951668 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2499754460e04c27ac53d52a4a67f557" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:08.955329 10365 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1acc16e2d3e14ab092da083b720cc29a has no consensus state
I20260924 00:21:08.956297 10365 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_a51a7905e6f848edb48e25b18d0ae196"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:08.957530 10589 leader_election.cc:302] T a72742864a6b4299b71757fb308dcadf P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08
I20260924 00:21:08.959494 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2499754460e04c27ac53d52a4a67f557" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
W20260924 00:21:08.968178 10589 leader_election.cc:341] T 2499754460e04c27ac53d52a4a67f557 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:08.970395 11041 tablet_bootstrap.cc:654] T b36cbb6520de4df58b88cfa53c20ee7a P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
W20260924 00:21:08.974807 10365 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a51a7905e6f848edb48e25b18d0ae196 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:08.975335 11039 tablet_bootstrap.cc:654] T 0c583d9ba1414f8ea2e605b4b6b076f8 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
W20260924 00:21:08.977635 10592 leader_election.cc:341] T 2499754460e04c27ac53d52a4a67f557 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:08.981660 10592 leader_election.cc:302] T 2499754460e04c27ac53d52a4a67f557 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:08.982261 11605 raft_consensus.cc:2726] T 2499754460e04c27ac53d52a4a67f557 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:08.996073 11049 ts_tablet_manager.cc:1918] T 772fb3d6180e499daf94ceeb9b26048d P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:08.996474 11457 raft_consensus.cc:2781] T a72742864a6b4299b71757fb308dcadf P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:08.996981 11457 raft_consensus.cc:482] T a72742864a6b4299b71757fb308dcadf P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:08.997289 11457 raft_consensus.cc:3037] T a72742864a6b4299b71757fb308dcadf P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:08.970330 11606 raft_consensus.cc:504] T 72d3bb80e1034881b94eb75de5d900c2 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:09.002905 11606 leader_election.cc:288] T 72d3bb80e1034881b94eb75de5d900c2 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:09.004336 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72d3bb80e1034881b94eb75de5d900c2" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:09.005682 11610 raft_consensus.cc:482] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 00:21:09.005743 10589 leader_election.cc:341] T 72d3bb80e1034881b94eb75de5d900c2 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:09.006075 11610 raft_consensus.cc:504] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:09.007736 11610 leader_election.cc:288] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
W20260924 00:21:09.009649 10876 leader_election.cc:341] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: 95a7a1d2147c4d7ea63ca37a8f0ea4e6
I20260924 00:21:09.014333 10794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72d3bb80e1034881b94eb75de5d900c2" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:09.015698 10592 leader_election.cc:341] T 72d3bb80e1034881b94eb75de5d900c2 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:09.015995 11040 tablet_bootstrap.cc:492] T 18abdec265a24dc8bfc765d4395767c0 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:09.016340 10592 leader_election.cc:302] T 72d3bb80e1034881b94eb75de5d900c2 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:09.018194 11606 raft_consensus.cc:2726] T 72d3bb80e1034881b94eb75de5d900c2 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:09.016755 11049 log.cc:1196] T 772fb3d6180e499daf94ceeb9b26048d P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/772fb3d6180e499daf94ceeb9b26048d
I20260924 00:21:09.019869 10845 heartbeater.cc:347] Connected to a master server at 127.10.4.124:35345
I20260924 00:21:09.021374 11049 ts_tablet_manager.cc:1939] T 772fb3d6180e499daf94ceeb9b26048d P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:09.016389 11040 ts_tablet_manager.cc:1397] T 18abdec265a24dc8bfc765d4395767c0 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.433s user 0.004s sys 0.014s
I20260924 00:21:09.028824 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 4420cead1f284af8958f189557833098 (table default.Table_bdbcdcde9ac241d5b22b5e8b0da6a9a7 [id=21e1b3346ba346c883fced1e7a1b6cf6]) successfully deleted
W20260924 00:21:09.035636 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 84e73180fab74524b4d568a010dc7175 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 84e73180fab74524b4d568a010dc7175 already in progress: creating tablet
W20260924 00:21:09.037175 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 915e8072851c4d2b90f8af12b1dc58e0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 915e8072851c4d2b90f8af12b1dc58e0 already in progress: creating tablet
I20260924 00:21:09.039733 10367 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_8a934fde5e054e3eab58cfb899302830" } new_table_name: "default.Table_3e21dcff4f0b40299e8ddb8ebb7f3c22" modify_external_catalogs: true
I20260924 00:21:09.039844 11040 raft_consensus.cc:348] T 18abdec265a24dc8bfc765d4395767c0 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:09.040457 11040 raft_consensus.cc:374] T 18abdec265a24dc8bfc765d4395767c0 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:09.045960 10794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95a7a1d2147c4d7ea63ca37a8f0ea4e6" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:09.040683 11040 raft_consensus.cc:729] T 18abdec265a24dc8bfc765d4395767c0 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:09.047436 11040 consensus_queue.cc:260] T 18abdec265a24dc8bfc765d4395767c0 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:09.049309 11040 ts_tablet_manager.cc:1428] T 18abdec265a24dc8bfc765d4395767c0 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.023s user 0.005s sys 0.000s
I20260924 00:21:09.050283 11049 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 233) took 1191 ms. Trace:
I20260924 00:21:09.051337 11049 rpcz_store.cc:276] 0924 00:21:07.858864 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:07.859487 (+ 623us) service_pool.cc:227] Handling call
0924 00:21:08.732975 (+873488us) ts_tablet_manager.cc:1145] Deleting tablet 772fb3d6180e499daf94ceeb9b26048d
0924 00:21:08.733017 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:08.942876 (+209859us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:08.995869 (+ 52993us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:09.050229 (+ 54360us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":872284}
I20260924 00:21:09.053900 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 772fb3d6180e499daf94ceeb9b26048d (table default.Table_9c2aa717e62340c785be379c0dd5837c [id=737634d9f0f4413c8b6dcaa26de87e2d]) successfully deleted
I20260924 00:21:09.055066 11049 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 235) took 1133 ms. Trace:
I20260924 00:21:09.056151 11049 rpcz_store.cc:276] 0924 00:21:07.922135 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:07.928436 (+ 6301us) service_pool.cc:227] Handling call
0924 00:21:09.054738 (+1126302us) ts_tablet_manager.cc:1145] Deleting tablet e46f92f2a020402681aa597e57f37c41
0924 00:21:09.054778 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:09.055018 (+ 240us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1101674}
W20260924 00:21:09.057976 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet e46f92f2a020402681aa597e57f37c41 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e46f92f2a020402681aa597e57f37c41 already in progress: creating tablet
I20260924 00:21:09.057787 11622 tablet.cc:1722] T 85a0ad61b7cb4e11b30e84ddc4655b64 P 48fbd603abe146c585d15196e1b5be08: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260924 00:21:09.050390 11457 raft_consensus.cc:504] T a72742864a6b4299b71757fb308dcadf P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:09.060563 10794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a72742864a6b4299b71757fb308dcadf" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
W20260924 00:21:09.062052 10592 leader_election.cc:341] T a72742864a6b4299b71757fb308dcadf P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
W20260924 00:21:09.062955 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 79d8dc7403f448df8cb960311eefe110 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 79d8dc7403f448df8cb960311eefe110 already in progress: creating tablet
W20260924 00:21:09.066311 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 84e73180fab74524b4d568a010dc7175 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 84e73180fab74524b4d568a010dc7175 already in progress: creating tablet
I20260924 00:21:09.067204 11041 tablet_bootstrap.cc:492] T b36cbb6520de4df58b88cfa53c20ee7a P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:09.067600 11041 ts_tablet_manager.cc:1397] T b36cbb6520de4df58b88cfa53c20ee7a P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.132s user 0.011s sys 0.002s
I20260924 00:21:09.068388 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a72742864a6b4299b71757fb308dcadf" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef"
I20260924 00:21:09.068922 10942 raft_consensus.cc:3037] T a72742864a6b4299b71757fb308dcadf P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:09.069669 11041 raft_consensus.cc:348] T b36cbb6520de4df58b88cfa53c20ee7a P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:09.070299 11041 raft_consensus.cc:374] T b36cbb6520de4df58b88cfa53c20ee7a P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:09.070526 11041 raft_consensus.cc:729] T b36cbb6520de4df58b88cfa53c20ee7a P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:09.071060 11041 consensus_queue.cc:260] T b36cbb6520de4df58b88cfa53c20ee7a P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:09.072835 11041 ts_tablet_manager.cc:1428] T b36cbb6520de4df58b88cfa53c20ee7a P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260924 00:21:09.078006 10942 raft_consensus.cc:2454] T a72742864a6b4299b71757fb308dcadf P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b10a9082a37b46f09d581944914c1670 in term 1.
W20260924 00:21:09.081139 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 915e8072851c4d2b90f8af12b1dc58e0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 915e8072851c4d2b90f8af12b1dc58e0 already in progress: creating tablet
I20260924 00:21:09.079154 10589 leader_election.cc:302] T a72742864a6b4299b71757fb308dcadf P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08
I20260924 00:21:09.077975 11041 tablet_bootstrap.cc:492] T 59a0a950a8144e99b2ed47be9a567d9a P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:09.085765 11606 raft_consensus.cc:2781] T a72742864a6b4299b71757fb308dcadf P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:09.086591 11606 raft_consensus.cc:686] T a72742864a6b4299b71757fb308dcadf P b10a9082a37b46f09d581944914c1670 [term 1 LEADER]: Becoming Leader. State: Replica: b10a9082a37b46f09d581944914c1670, State: Running, Role: LEADER
I20260924 00:21:09.087366 11606 consensus_queue.cc:237] T a72742864a6b4299b71757fb308dcadf P b10a9082a37b46f09d581944914c1670 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:09.090418 11040 tablet_bootstrap.cc:492] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:09.079370 11457 leader_election.cc:288] T a72742864a6b4299b71757fb308dcadf P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Requested vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
W20260924 00:21:09.095002 10878 leader_election.cc:341] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:09.095474 10878 leader_election.cc:302] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670
I20260924 00:21:09.095906 11039 tablet_bootstrap.cc:492] T 0c583d9ba1414f8ea2e605b4b6b076f8 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:09.096197 11610 raft_consensus.cc:2726] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:09.096364 11039 ts_tablet_manager.cc:1397] T 0c583d9ba1414f8ea2e605b4b6b076f8 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.165s user 0.003s sys 0.008s
I20260924 00:21:09.098706 10371 master_service.cc:946] ReplaceTablet: received request to replace tablet df24a60e710241de9110afc5a6167bf9 from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:09.098592 11039 raft_consensus.cc:348] T 0c583d9ba1414f8ea2e605b4b6b076f8 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:09.099278 11039 raft_consensus.cc:374] T 0c583d9ba1414f8ea2e605b4b6b076f8 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:09.099511 11039 raft_consensus.cc:729] T 0c583d9ba1414f8ea2e605b4b6b076f8 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:09.102807 11039 consensus_queue.cc:260] T 0c583d9ba1414f8ea2e605b4b6b076f8 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:09.106458 10701 heartbeater.cc:347] Connected to a master server at 127.10.4.124:35345
I20260924 00:21:09.112512 11039 ts_tablet_manager.cc:1428] T 0c583d9ba1414f8ea2e605b4b6b076f8 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.016s user 0.003s sys 0.001s
I20260924 00:21:09.107209 11041 tablet_bootstrap.cc:654] T 59a0a950a8144e99b2ed47be9a567d9a P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:09.117674 11039 tablet_bootstrap.cc:492] T a0f71049f6b3478da3b07ebc9293ce64 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:09.128139 10374 catalog_manager.cc:5495] T bf144ab1a6464a4ab02ac0642c5255e3 P 48fbd603abe146c585d15196e1b5be08 reported cstate change: term changed from 0 to 1, leader changed from <none> to 48fbd603abe146c585d15196e1b5be08 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "48fbd603abe146c585d15196e1b5be08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } health_report { overall_health: UNKNOWN } } }
I20260924 00:21:09.129403 10374 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet bf144ab1a6464a4ab02ac0642c5255e3 (table default.Table_382099cf77a24bb8bd6fe084926431c9 [id=3db896d304514ead8dfe726c402a6fcd]). Expected version 1 got 0
I20260924 00:21:09.132134 10774 tablet_service.cc:1467] Processing CreateTablet for tablet bc89341cd96e4c498160fb1412715eaa (DEFAULT_TABLE table=default.Table_6ba8cc42d97e4576a8c2021b39b78dbb [id=6ac68d6b800747d59768d4988137dae7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.132979 10366 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 18abdec265a24dc8bfc765d4395767c0 (table default.Table_a5a83435a7fb47ce8083022c0686b74a [id=a587a50b0c204843a41acacc45c2b150]). Expected version 1 got 0
I20260924 00:21:09.133148 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc89341cd96e4c498160fb1412715eaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.133585 10366 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 119d83562ef546cab496cd7b44fecc15 (table default.Table_76ffb857cf034b6f80598edd90dad953 [id=5fb5e1e90c9440f385e7b04520bcba9c]). Expected version 1 got 0
I20260924 00:21:09.149497 10777 tablet_service.cc:1467] Processing CreateTablet for tablet 1a7eb76a787a447793fd04686f748d1e (DEFAULT_TABLE table=default.Table_c6e518c165b3408d80c30626859c2086 [id=0c60980118d24a7ab25ec4e75f8dd9df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.150465 10777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a7eb76a787a447793fd04686f748d1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.151490 10775 tablet_service.cc:1467] Processing CreateTablet for tablet 3dae8b8d7743416983e9e903ea57140d (DEFAULT_TABLE table=default.Table_e735211dc38c444da1b3af42f354c740 [id=8031d25aceec4ca48e8b29e7d75e5320]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.152344 10775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3dae8b8d7743416983e9e903ea57140d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.153793 10778 tablet_service.cc:1467] Processing CreateTablet for tablet fe43761fcbd6415c8c250a4c419f7aa2 (DEFAULT_TABLE table=default.Table_42a1cf1e668747c88d6179f35dd0e620 [id=9839446621034266a1e4815b8f4863c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.154165 10920 tablet_service.cc:1467] Processing CreateTablet for tablet bc89341cd96e4c498160fb1412715eaa (DEFAULT_TABLE table=default.Table_6ba8cc42d97e4576a8c2021b39b78dbb [id=6ac68d6b800747d59768d4988137dae7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.154611 10778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe43761fcbd6415c8c250a4c419f7aa2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.155076 10920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc89341cd96e4c498160fb1412715eaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.157934 10909 tablet_service.cc:1467] Processing CreateTablet for tablet 1a7eb76a787a447793fd04686f748d1e (DEFAULT_TABLE table=default.Table_c6e518c165b3408d80c30626859c2086 [id=0c60980118d24a7ab25ec4e75f8dd9df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.159374 10912 tablet_service.cc:1467] Processing CreateTablet for tablet 3dae8b8d7743416983e9e903ea57140d (DEFAULT_TABLE table=default.Table_e735211dc38c444da1b3af42f354c740 [id=8031d25aceec4ca48e8b29e7d75e5320]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.161650 10909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a7eb76a787a447793fd04686f748d1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.162129 10365 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_42a1cf1e668747c88d6179f35dd0e620" } new_table_name: "default.Table_29d22b606c3a4d4ba2fce79bd410c4f4" modify_external_catalogs: true
I20260924 00:21:09.162334 10625 tablet_service.cc:1467] Processing CreateTablet for tablet bc89341cd96e4c498160fb1412715eaa (DEFAULT_TABLE table=default.Table_6ba8cc42d97e4576a8c2021b39b78dbb [id=6ac68d6b800747d59768d4988137dae7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.163405 10912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3dae8b8d7743416983e9e903ea57140d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.174322 10634 tablet_service.cc:1467] Processing CreateTablet for tablet 1a7eb76a787a447793fd04686f748d1e (DEFAULT_TABLE table=default.Table_c6e518c165b3408d80c30626859c2086 [id=0c60980118d24a7ab25ec4e75f8dd9df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.176069 10370 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 119d83562ef546cab496cd7b44fecc15 (table default.Table_76ffb857cf034b6f80598edd90dad953 [id=5fb5e1e90c9440f385e7b04520bcba9c]). Expected version 1 got 0
I20260924 00:21:09.176232 10628 tablet_service.cc:1467] Processing CreateTablet for tablet 3dae8b8d7743416983e9e903ea57140d (DEFAULT_TABLE table=default.Table_e735211dc38c444da1b3af42f354c740 [id=8031d25aceec4ca48e8b29e7d75e5320]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.177019 10368 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 60ebab3e93f54383b87aeb54bbde722d
I20260924 00:21:09.177861 10625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc89341cd96e4c498160fb1412715eaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.178361 10368 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:47022 (request call id 300) took 1054 ms. Trace:
I20260924 00:21:09.181466 10634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a7eb76a787a447793fd04686f748d1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.182960 10628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3dae8b8d7743416983e9e903ea57140d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:21:09.183418 10370 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.4.65:55561 (request call id 62) took 988 ms (client timeout 1000 ms). Trace:
W20260924 00:21:09.183712 10370 rpcz_store.cc:269] 0924 00:21:08.195464 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:08.206872 (+ 11408us) service_pool.cc:227] Handling call
0924 00:21:09.183359 (+976487us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":33106,"reported_tablets":2}
I20260924 00:21:09.187032 10780 tablet_service.cc:1514] Processing DeleteTablet for tablet 60ebab3e93f54383b87aeb54bbde722d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:08 UTC) from {username='slave'} at 127.0.0.1:35996
W20260924 00:21:09.198628 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 60ebab3e93f54383b87aeb54bbde722d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 60ebab3e93f54383b87aeb54bbde722d already in progress: creating tablet
I20260924 00:21:09.178615 10368 rpcz_store.cc:276] 0924 00:21:08.123845 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:08.123941 (+ 96us) service_pool.cc:227] Handling call
0924 00:21:08.125290 (+ 1349us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0924 00:21:08.126612 (+ 1322us) catalog_manager.cc:2752] Modifying in-memory table state
0924 00:21:08.126769 (+ 157us) catalog_manager.cc:2761] Locking tablets
0924 00:21:08.127356 (+ 587us) catalog_manager.cc:2775] Removing table and tablets from system table
0924 00:21:08.484581 (+357225us) catalog_manager.cc:2793] Removing table from by-name map
0924 00:21:08.484778 (+ 197us) catalog_manager.cc:2809] Committing in-memory state
0924 00:21:09.176916 (+692138us) catalog_manager.cc:2813] Aborting table tasks
0924 00:21:09.178307 (+ 1391us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 00:21:08.312227 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 00:21:08.312526 (+ 299us) write_op.cc:432] Acquiring schema lock in shared mode
0924 00:21:08.312545 (+ 19us) write_op.cc:435] Acquired schema lock
0924 00:21:08.312556 (+ 11us) tablet.cc:660] Decoding operations
0924 00:21:08.312808 (+ 252us) write_op.cc:620] Acquiring the partition lock for write op
0924 00:21:08.312849 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0924 00:21:08.312869 (+ 20us) tablet.cc:683] Acquiring locks for 2 operations
0924 00:21:08.329242 (+ 16373us) tablet.cc:699] Row locks acquired
0924 00:21:08.329258 (+ 16us) write_op.cc:260] PREPARE: finished
0924 00:21:08.329363 (+ 105us) write_op.cc:270] Start()
0924 00:21:08.333839 (+ 4476us) write_op.cc:276] Timestamp: P: 1790209268329328 usec, L: 0
0924 00:21:08.333862 (+ 23us) op_driver.cc:331] REPLICATION: starting
0924 00:21:08.334349 (+ 487us) log.cc:842] Serialized 756 byte log entry
0924 00:21:08.481766 (+147417us) op_driver.cc:444] REPLICATION: finished
0924 00:21:08.482824 (+ 1058us) write_op.cc:301] APPLY: starting
0924 00:21:08.482871 (+ 47us) tablet.cc:1365] starting BulkCheckPresence
0924 00:21:08.482979 (+ 108us) tablet.cc:1368] finished BulkCheckPresence
0924 00:21:08.483075 (+ 96us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 00:21:08.483275 (+ 200us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 00:21:08.483395 (+ 120us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0924 00:21:08.483437 (+ 42us) write_op.cc:312] APPLY: finished
0924 00:21:08.483581 (+ 144us) log.cc:842] Serialized 29 byte log entry
0924 00:21:08.483696 (+ 115us) write_op.cc:489] Releasing partition, row and schema locks
0924 00:21:08.483750 (+ 54us) write_op.cc:454] Released schema lock
0924 00:21:08.483777 (+ 27us) write_op.cc:341] FINISH: Updating metrics
0924 00:21:08.490263 (+ 6486us) write_op.cc:489] Releasing partition, row and schema locks
0924 00:21:08.490288 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":947,"apply.run_cpu_time_us":1365,"apply.run_wall_time_us":7630,"num_ops":2,"prepare.queue_time_us":183205,"prepare.run_cpu_time_us":1885,"prepare.run_wall_time_us":22306,"replication_time_us":147842}]]}
I20260924 00:21:09.206338 11040 tablet_bootstrap.cc:654] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:09.223538 10915 tablet_service.cc:1514] Processing DeleteTablet for tablet 4420cead1f284af8958f189557833098 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:01 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:09.229529 10627 tablet_service.cc:1514] Processing DeleteTablet for tablet 60ebab3e93f54383b87aeb54bbde722d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:08 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:09.230311 11049 tablet_replica.cc:330] T 60ebab3e93f54383b87aeb54bbde722d P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:09.230974 11049 raft_consensus.cc:2229] T 60ebab3e93f54383b87aeb54bbde722d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:09.231351 11049 raft_consensus.cc:2258] T 60ebab3e93f54383b87aeb54bbde722d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:09.232798 11049 ts_tablet_manager.cc:1905] T 60ebab3e93f54383b87aeb54bbde722d P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:09.237538 10909 tablet_service.cc:1514] Processing DeleteTablet for tablet 60ebab3e93f54383b87aeb54bbde722d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:08 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:09.240394 10915 tablet_service.cc:1514] Processing DeleteTablet for tablet 772fb3d6180e499daf94ceeb9b26048d with delete_type TABLET_DATA_DELETED (replaced by tablet 0856ff2896ce4832af348947f226ee7b) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:09.241212 10907 tablet_service.cc:1514] Processing DeleteTablet for tablet 95a7a1d2147c4d7ea63ca37a8f0ea4e6 with delete_type TABLET_DATA_DELETED (replaced by tablet 3b8d74761b0b4fd0a22021d3870ec05c) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:09.241959 11051 tablet_replica.cc:330] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:09.242637 11051 raft_consensus.cc:2229] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:09.243039 11051 raft_consensus.cc:2258] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus is shut down!
W20260924 00:21:09.234907 10367 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a62b85ebb2b84ae19cad3e3e2dd07a59 consensus state has no leader
W20260924 00:21:09.241631 10358 catalog_manager.cc:4808] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 4420cead1f284af8958f189557833098 because the tablet was not found. No further retry: Not found: Tablet not found: 4420cead1f284af8958f189557833098
I20260924 00:21:09.244553 11051 ts_tablet_manager.cc:1905] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260924 00:21:09.245018 10358 catalog_manager.cc:4808] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 772fb3d6180e499daf94ceeb9b26048d because the tablet was not found. No further retry: Not found: Tablet not found: 772fb3d6180e499daf94ceeb9b26048d
W20260924 00:21:09.216712 10366 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.4.67:49569 (request call id 59) took 1002 ms (client timeout 1000 ms). Trace:
W20260924 00:21:09.245698 10366 rpcz_store.cc:269] 0924 00:21:08.214478 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:08.214593 (+ 115us) service_pool.cc:227] Handling call
0924 00:21:09.216646 (+1002053us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":5}
I20260924 00:21:09.251896 10627 tablet_service.cc:1467] Processing CreateTablet for tablet fe43761fcbd6415c8c250a4c419f7aa2 (DEFAULT_TABLE table=default.Table_42a1cf1e668747c88d6179f35dd0e620 [id=9839446621034266a1e4815b8f4863c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.253005 10627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe43761fcbd6415c8c250a4c419f7aa2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.263520 11051 ts_tablet_manager.cc:1918] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:09.263856 11051 log.cc:1196] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/95a7a1d2147c4d7ea63ca37a8f0ea4e6
I20260924 00:21:09.264006 10366 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_e110cb436d444a319359e45630f8e2bb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:09.265017 10632 tablet_service.cc:1467] Processing CreateTablet for tablet e33db14b4aec4bcd8c6f16dd418a23fe (DEFAULT_TABLE table=default.Table_cae9fb8a84b248ac96ee0c3ee9874324 [id=972f6bb2a2094f5a9a49ac19ff3e0f71]), partition=RANGE (key) PARTITION UNBOUNDED
W20260924 00:21:09.265424 10366 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e110cb436d444a319359e45630f8e2bb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:09.266023 10632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e33db14b4aec4bcd8c6f16dd418a23fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.270836 10373 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 18abdec265a24dc8bfc765d4395767c0
I20260924 00:21:09.272128 10373 catalog_manager.cc:6565] ReplaceTablet: tablet 18abdec265a24dc8bfc765d4395767c0 deleted and replaced by tablet 45ba52ea61364b988ce77b265f8c7a79
I20260924 00:21:09.274793 10371 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet df24a60e710241de9110afc5a6167bf9
I20260924 00:21:09.275910 10371 catalog_manager.cc:6565] ReplaceTablet: tablet df24a60e710241de9110afc5a6167bf9 deleted and replaced by tablet fd0c3f766a464116aa777b849cf4acc2
I20260924 00:21:09.277863 10773 tablet_service.cc:1514] Processing DeleteTablet for tablet 18abdec265a24dc8bfc765d4395767c0 with delete_type TABLET_DATA_DELETED (replaced by tablet 45ba52ea61364b988ce77b265f8c7a79) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:09.278460 10779 tablet_service.cc:1514] Processing DeleteTablet for tablet df24a60e710241de9110afc5a6167bf9 with delete_type TABLET_DATA_DELETED (replaced by tablet fd0c3f766a464116aa777b849cf4acc2) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:09.278759 11051 ts_tablet_manager.cc:1939] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:09.279292 11335 tablet_replica.cc:330] T 18abdec265a24dc8bfc765d4395767c0 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:09.279934 11335 raft_consensus.cc:2229] T 18abdec265a24dc8bfc765d4395767c0 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:09.284381 10631 tablet_service.cc:1514] Processing DeleteTablet for tablet 18abdec265a24dc8bfc765d4395767c0 with delete_type TABLET_DATA_DELETED (replaced by tablet 45ba52ea61364b988ce77b265f8c7a79) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:09.285368 11051 tablet_replica.cc:330] T 60ebab3e93f54383b87aeb54bbde722d P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:09.286046 11051 raft_consensus.cc:2229] T 60ebab3e93f54383b87aeb54bbde722d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:09.286369 10631 tablet_service.cc:1514] Processing DeleteTablet for tablet df24a60e710241de9110afc5a6167bf9 with delete_type TABLET_DATA_DELETED (replaced by tablet fd0c3f766a464116aa777b849cf4acc2) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:09.286454 11051 raft_consensus.cc:2258] T 60ebab3e93f54383b87aeb54bbde722d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:09.288324 11051 ts_tablet_manager.cc:1905] T 60ebab3e93f54383b87aeb54bbde722d P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260924 00:21:09.291155 10365 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fe43761fcbd6415c8c250a4c419f7aa2 consensus state has no leader
I20260924 00:21:09.291896 10370 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 72d3bb80e1034881b94eb75de5d900c2 (table default.Table_10ca22fe85d94c4b836dc22d24343797 [id=ca942906de0f4019bbdf72ec886fe6fb]). Expected version 2 got 0
I20260924 00:21:09.292508 10370 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 5f2a32d5afff458fab79ffa106a98884 (table default.Table_bf61c7623556425688b99c1c373e4879 [id=4f70e833d7f74d91af0faa01ed52f78b]). Expected version 1 got 0
I20260924 00:21:09.294963 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 95a7a1d2147c4d7ea63ca37a8f0ea4e6 (table default.Table_0cc93ce7fcfb44089209422666978c62 [id=c0cd199f74494deda9d088345c4c90a3]) successfully deleted
I20260924 00:21:09.297093 10907 tablet_service.cc:1514] Processing DeleteTablet for tablet 18abdec265a24dc8bfc765d4395767c0 with delete_type TABLET_DATA_DELETED (replaced by tablet 45ba52ea61364b988ce77b265f8c7a79) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:09.298970 10909 tablet_service.cc:1467] Processing CreateTablet for tablet e33db14b4aec4bcd8c6f16dd418a23fe (DEFAULT_TABLE table=default.Table_cae9fb8a84b248ac96ee0c3ee9874324 [id=972f6bb2a2094f5a9a49ac19ff3e0f71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.299901 10909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e33db14b4aec4bcd8c6f16dd418a23fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.301158 10915 tablet_service.cc:1514] Processing DeleteTablet for tablet df24a60e710241de9110afc5a6167bf9 with delete_type TABLET_DATA_DELETED (replaced by tablet fd0c3f766a464116aa777b849cf4acc2) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:09.303907 10365 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_b5c9933d8e0c481290863b60a642e8e7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:09.305276 10365 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b5c9933d8e0c481290863b60a642e8e7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:09.305181 10920 tablet_service.cc:1467] Processing CreateTablet for tablet fe43761fcbd6415c8c250a4c419f7aa2 (DEFAULT_TABLE table=default.Table_42a1cf1e668747c88d6179f35dd0e620 [id=9839446621034266a1e4815b8f4863c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.309191 10374 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_9b48f9449d9149128f9d014b582394ac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:09.309937 10920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe43761fcbd6415c8c250a4c419f7aa2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:21:09.310599 10374 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9b48f9449d9149128f9d014b582394ac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:09.314792 11049 ts_tablet_manager.cc:1918] T 60ebab3e93f54383b87aeb54bbde722d P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:09.315121 11049 log.cc:1196] T 60ebab3e93f54383b87aeb54bbde722d P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/60ebab3e93f54383b87aeb54bbde722d
I20260924 00:21:09.315687 11049 ts_tablet_manager.cc:1939] T 60ebab3e93f54383b87aeb54bbde722d P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:09.317870 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 60ebab3e93f54383b87aeb54bbde722d (table default.Table_88117cdd98a74bcfa88b82c3162e7119 [id=ed6b4628a295498da00f778ad37db72e]) successfully deleted
I20260924 00:21:09.319643 10628 tablet_service.cc:1467] Processing CreateTablet for tablet 0c7b7a63ed8f44abb9872d36f756114d (DEFAULT_TABLE table=default.Table_6453acb2750442d0a172188dbc89d856 [id=0dd7e590773f4405bfb9c2a5060b3413]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.320632 10628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c7b7a63ed8f44abb9872d36f756114d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.320889 10915 tablet_service.cc:1467] Processing CreateTablet for tablet 0c7b7a63ed8f44abb9872d36f756114d (DEFAULT_TABLE table=default.Table_6453acb2750442d0a172188dbc89d856 [id=0dd7e590773f4405bfb9c2a5060b3413]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.321887 10915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c7b7a63ed8f44abb9872d36f756114d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.328231 11039 tablet_bootstrap.cc:654] T a0f71049f6b3478da3b07ebc9293ce64 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:09.332793 10780 tablet_service.cc:1467] Processing CreateTablet for tablet e33db14b4aec4bcd8c6f16dd418a23fe (DEFAULT_TABLE table=default.Table_cae9fb8a84b248ac96ee0c3ee9874324 [id=972f6bb2a2094f5a9a49ac19ff3e0f71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.333736 10780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e33db14b4aec4bcd8c6f16dd418a23fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.333760 11049 tablet_replica.cc:330] T 18abdec265a24dc8bfc765d4395767c0 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:09.336266 10779 tablet_service.cc:1467] Processing CreateTablet for tablet 0c7b7a63ed8f44abb9872d36f756114d (DEFAULT_TABLE table=default.Table_6453acb2750442d0a172188dbc89d856 [id=0dd7e590773f4405bfb9c2a5060b3413]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.337242 11335 raft_consensus.cc:2258] T 18abdec265a24dc8bfc765d4395767c0 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:09.338858 10775 tablet_service.cc:1467] Processing CreateTablet for tablet 4aa0dc5c502a406b99609f044b6baeb0 (DEFAULT_TABLE table=default.Table_82b79ff8ffca4a7880702b55cd10e393 [id=7e160144b1704e27a1d7f729c7a04049]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.339061 11335 ts_tablet_manager.cc:1905] T 18abdec265a24dc8bfc765d4395767c0 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:09.334455 11049 raft_consensus.cc:2229] T 18abdec265a24dc8bfc765d4395767c0 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:09.345432 11049 raft_consensus.cc:2258] T 18abdec265a24dc8bfc765d4395767c0 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:09.346576 10775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4aa0dc5c502a406b99609f044b6baeb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.346971 11049 ts_tablet_manager.cc:1905] T 18abdec265a24dc8bfc765d4395767c0 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:09.352933 10779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c7b7a63ed8f44abb9872d36f756114d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.355093 11359 consensus_queue.cc:1035] T bf144ab1a6464a4ab02ac0642c5255e3 P 48fbd603abe146c585d15196e1b5be08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:09.356621 11359 consensus_queue.cc:1035] T bf144ab1a6464a4ab02ac0642c5255e3 P 48fbd603abe146c585d15196e1b5be08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:09.369829 11335 ts_tablet_manager.cc:1918] T 18abdec265a24dc8bfc765d4395767c0 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:09.370193 11335 log.cc:1196] T 18abdec265a24dc8bfc765d4395767c0 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/18abdec265a24dc8bfc765d4395767c0
I20260924 00:21:09.370734 11335 ts_tablet_manager.cc:1939] T 18abdec265a24dc8bfc765d4395767c0 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:09.372823 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 18abdec265a24dc8bfc765d4395767c0 (table default.Table_a5a83435a7fb47ce8083022c0686b74a [id=a587a50b0c204843a41acacc45c2b150]) successfully deleted
W20260924 00:21:09.378149 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet df24a60e710241de9110afc5a6167bf9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet df24a60e710241de9110afc5a6167bf9 already in progress: creating tablet
I20260924 00:21:09.384117 11049 ts_tablet_manager.cc:1918] T 18abdec265a24dc8bfc765d4395767c0 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:09.384439 11049 log.cc:1196] T 18abdec265a24dc8bfc765d4395767c0 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/18abdec265a24dc8bfc765d4395767c0
I20260924 00:21:09.384994 11049 ts_tablet_manager.cc:1939] T 18abdec265a24dc8bfc765d4395767c0 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:09.386013 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 18abdec265a24dc8bfc765d4395767c0 (table default.Table_a5a83435a7fb47ce8083022c0686b74a [id=a587a50b0c204843a41acacc45c2b150]) successfully deleted
I20260924 00:21:09.393384 10367 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 119d83562ef546cab496cd7b44fecc15 (table default.Table_76ffb857cf034b6f80598edd90dad953 [id=5fb5e1e90c9440f385e7b04520bcba9c]). Expected version 1 got 0
I20260924 00:21:09.394544 10367 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 5f2a32d5afff458fab79ffa106a98884 (table default.Table_bf61c7623556425688b99c1c373e4879 [id=4f70e833d7f74d91af0faa01ed52f78b]). Expected version 1 got 0
I20260924 00:21:09.395224 10367 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet bf144ab1a6464a4ab02ac0642c5255e3 (table default.Table_382099cf77a24bb8bd6fe084926431c9 [id=3db896d304514ead8dfe726c402a6fcd]). Expected version 1 got 0
I20260924 00:21:09.396394 10369 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 4325b60efc1649c9ba4828d0a098a676
I20260924 00:21:09.397873 10369 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:47022 (request call id 303) took 1118 ms. Trace:
I20260924 00:21:09.398144 10369 rpcz_store.cc:276] 0924 00:21:08.279453 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:08.318744 (+ 39291us) service_pool.cc:227] Handling call
0924 00:21:08.343520 (+ 24776us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0924 00:21:08.344246 (+ 726us) catalog_manager.cc:2752] Modifying in-memory table state
0924 00:21:08.344477 (+ 231us) catalog_manager.cc:2761] Locking tablets
0924 00:21:09.125634 (+781157us) catalog_manager.cc:2775] Removing table and tablets from system table
0924 00:21:09.267650 (+142016us) catalog_manager.cc:2793] Removing table from by-name map
0924 00:21:09.267852 (+ 202us) catalog_manager.cc:2809] Committing in-memory state
0924 00:21:09.396305 (+128453us) catalog_manager.cc:2813] Aborting table tasks
0924 00:21:09.397823 (+ 1518us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 00:21:09.127508 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 00:21:09.127843 (+ 335us) write_op.cc:432] Acquiring schema lock in shared mode
0924 00:21:09.127865 (+ 22us) write_op.cc:435] Acquired schema lock
0924 00:21:09.127876 (+ 11us) tablet.cc:660] Decoding operations
0924 00:21:09.128173 (+ 297us) write_op.cc:620] Acquiring the partition lock for write op
0924 00:21:09.128207 (+ 34us) write_op.cc:641] Partition lock acquired for write op
0924 00:21:09.128227 (+ 20us) tablet.cc:683] Acquiring locks for 2 operations
0924 00:21:09.128504 (+ 277us) tablet.cc:699] Row locks acquired
0924 00:21:09.128517 (+ 13us) write_op.cc:260] PREPARE: finished
0924 00:21:09.128618 (+ 101us) write_op.cc:270] Start()
0924 00:21:09.128707 (+ 89us) write_op.cc:276] Timestamp: P: 1790209269128542 usec, L: 0
0924 00:21:09.128736 (+ 29us) op_driver.cc:331] REPLICATION: starting
0924 00:21:09.129244 (+ 508us) log.cc:842] Serialized 756 byte log entry
0924 00:21:09.265338 (+136094us) op_driver.cc:444] REPLICATION: finished
0924 00:21:09.266273 (+ 935us) write_op.cc:301] APPLY: starting
0924 00:21:09.266329 (+ 56us) tablet.cc:1365] starting BulkCheckPresence
0924 00:21:09.266426 (+ 97us) tablet.cc:1368] finished BulkCheckPresence
0924 00:21:09.266529 (+ 103us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 00:21:09.266811 (+ 282us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 00:21:09.266951 (+ 140us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0924 00:21:09.266992 (+ 41us) write_op.cc:312] APPLY: finished
0924 00:21:09.267173 (+ 181us) log.cc:842] Serialized 29 byte log entry
0924 00:21:09.267314 (+ 141us) write_op.cc:489] Releasing partition, row and schema locks
0924 00:21:09.267376 (+ 62us) write_op.cc:454] Released schema lock
0924 00:21:09.267408 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0924 00:21:09.267561 (+ 153us) write_op.cc:489] Releasing partition, row and schema locks
0924 00:21:09.267582 (+ 21us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":786,"apply.run_cpu_time_us":1487,"apply.run_wall_time_us":1487,"num_ops":2,"prepare.queue_time_us":102,"prepare.run_cpu_time_us":1907,"prepare.run_wall_time_us":1914,"replication_time_us":136527}]]}
W20260924 00:21:09.401511 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet df24a60e710241de9110afc5a6167bf9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet df24a60e710241de9110afc5a6167bf9 already in progress: creating tablet
I20260924 00:21:09.401849 10628 tablet_service.cc:1514] Processing DeleteTablet for tablet 4325b60efc1649c9ba4828d0a098a676 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:08 UTC) from {username='slave'} at 127.0.0.1:46896
W20260924 00:21:09.402022 10732 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:41301, user_credentials={real_user=slave}} blocked reactor thread for 47274.3us
I20260924 00:21:09.402493 11049 tablet_replica.cc:330] T 4325b60efc1649c9ba4828d0a098a676 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:09.403154 11049 raft_consensus.cc:2229] T 4325b60efc1649c9ba4828d0a098a676 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:09.403584 11049 raft_consensus.cc:2258] T 4325b60efc1649c9ba4828d0a098a676 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:09.404644 10369 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_87fc5b8981644a5d862af47154aaa22a" } modify_external_catalogs: true
I20260924 00:21:09.405223 11049 ts_tablet_manager.cc:1905] T 4325b60efc1649c9ba4828d0a098a676 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:09.405143 10369 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_87fc5b8981644a5d862af47154aaa22a" } modify_external_catalogs: true
I20260924 00:21:09.407226 10371 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet bf144ab1a6464a4ab02ac0642c5255e3 (table default.Table_382099cf77a24bb8bd6fe084926431c9 [id=3db896d304514ead8dfe726c402a6fcd]). Expected version 1 got 0
I20260924 00:21:09.407828 10371 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet 5f2a32d5afff458fab79ffa106a98884 (table default.Table_bf61c7623556425688b99c1c373e4879 [id=4f70e833d7f74d91af0faa01ed52f78b]). Expected version 1 got 0
I20260924 00:21:09.425056 10778 tablet_service.cc:1514] Processing DeleteTablet for tablet 4325b60efc1649c9ba4828d0a098a676 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:08 UTC) from {username='slave'} at 127.0.0.1:35996
W20260924 00:21:09.426596 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 4325b60efc1649c9ba4828d0a098a676 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4325b60efc1649c9ba4828d0a098a676 already in progress: creating tablet
I20260924 00:21:09.437561 10373 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_14437b13dbe34d5591c2a180c12c23b1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:09.439062 10373 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_14437b13dbe34d5591c2a180c12c23b1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:09.441975 11049 ts_tablet_manager.cc:1918] T 4325b60efc1649c9ba4828d0a098a676 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:09.442317 11049 log.cc:1196] T 4325b60efc1649c9ba4828d0a098a676 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/4325b60efc1649c9ba4828d0a098a676
I20260924 00:21:09.442895 11049 ts_tablet_manager.cc:1939] T 4325b60efc1649c9ba4828d0a098a676 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:09.445220 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 4325b60efc1649c9ba4828d0a098a676 (table default.Table_4a5edca73a094b4d93ba4d389679ab48 [id=7a76983970cd491a8e8635db379bdd10]) successfully deleted
I20260924 00:21:09.453794 10372 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_08f63e23539246519606297527c945af" } new_table_name: "default.Table_cf2f09bc8e6849848a9e641521c37450" modify_external_catalogs: true
W20260924 00:21:09.461704 10878 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:35359, user_credentials={real_user=slave}} blocked reactor thread for 74217.9us
I20260924 00:21:09.466411 11606 raft_consensus.cc:482] T b36cbb6520de4df58b88cfa53c20ee7a P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:09.466930 11606 raft_consensus.cc:504] T b36cbb6520de4df58b88cfa53c20ee7a P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:09.468674 11606 leader_election.cc:288] T b36cbb6520de4df58b88cfa53c20ee7a P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:09.471931 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b36cbb6520de4df58b88cfa53c20ee7a" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:09.473269 10592 leader_election.cc:341] T b36cbb6520de4df58b88cfa53c20ee7a P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:09.474083 10939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b36cbb6520de4df58b88cfa53c20ee7a" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
W20260924 00:21:09.475344 10589 leader_election.cc:341] T b36cbb6520de4df58b88cfa53c20ee7a P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:09.475684 10589 leader_election.cc:302] T b36cbb6520de4df58b88cfa53c20ee7a P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:09.476615 11606 raft_consensus.cc:2726] T b36cbb6520de4df58b88cfa53c20ee7a P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:09.492373 10628 tablet_service.cc:1467] Processing CreateTablet for tablet 4aa0dc5c502a406b99609f044b6baeb0 (DEFAULT_TABLE table=default.Table_82b79ff8ffca4a7880702b55cd10e393 [id=7e160144b1704e27a1d7f729c7a04049]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.493428 10628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4aa0dc5c502a406b99609f044b6baeb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.498735 10368 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_5e4d36ef35c84b2ba0dcf2eec1b33822" } new_table_name: "default.Table_a9fb0b9afed748b38640e91ac8eeedd2" modify_external_catalogs: true
I20260924 00:21:09.500512 10915 tablet_service.cc:1514] Processing DeleteTablet for tablet 4325b60efc1649c9ba4828d0a098a676 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:08 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:09.505052 11041 tablet_bootstrap.cc:492] T 59a0a950a8144e99b2ed47be9a567d9a P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:09.505046 11051 ts_tablet_manager.cc:1918] T 60ebab3e93f54383b87aeb54bbde722d P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:09.505393 11051 log.cc:1196] T 60ebab3e93f54383b87aeb54bbde722d P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/60ebab3e93f54383b87aeb54bbde722d
I20260924 00:21:09.505409 11041 ts_tablet_manager.cc:1397] T 59a0a950a8144e99b2ed47be9a567d9a P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.428s user 0.012s sys 0.002s
I20260924 00:21:09.505971 11051 ts_tablet_manager.cc:1939] T 60ebab3e93f54383b87aeb54bbde722d P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:09.507974 11051 tablet_replica.cc:330] T 18abdec265a24dc8bfc765d4395767c0 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:09.508596 11051 raft_consensus.cc:2229] T 18abdec265a24dc8bfc765d4395767c0 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:21:09.509671 11040 tablet_bootstrap.cc:492] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:09.509799 11051 pending_rounds.cc:62] T 18abdec265a24dc8bfc765d4395767c0 P c8626e300d0644b0908797ad61bd7cef: Trying to abort 1 pending ops.
I20260924 00:21:09.509984 11051 pending_rounds.cc:68] T 18abdec265a24dc8bfc765d4395767c0 P c8626e300d0644b0908797ad61bd7cef: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332697164470267904 op_type: NO_OP noop_request { }
I20260924 00:21:09.510490 11051 raft_consensus.cc:2866] T 18abdec265a24dc8bfc765d4395767c0 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260924 00:21:09.510778 11051 raft_consensus.cc:2258] T 18abdec265a24dc8bfc765d4395767c0 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:09.513273 11051 ts_tablet_manager.cc:1905] T 18abdec265a24dc8bfc765d4395767c0 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:09.523910 11040 ts_tablet_manager.cc:1397] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.434s user 0.014s sys 0.005s
I20260924 00:21:09.526335 11051 ts_tablet_manager.cc:1918] T 18abdec265a24dc8bfc765d4395767c0 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:09.526638 11051 log.cc:1196] T 18abdec265a24dc8bfc765d4395767c0 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/18abdec265a24dc8bfc765d4395767c0
I20260924 00:21:09.527268 11051 ts_tablet_manager.cc:1939] T 18abdec265a24dc8bfc765d4395767c0 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:09.527176 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 60ebab3e93f54383b87aeb54bbde722d (table default.Table_88117cdd98a74bcfa88b82c3162e7119 [id=ed6b4628a295498da00f778ad37db72e]) successfully deleted
I20260924 00:21:09.528573 11051 tablet_replica.cc:330] T 4325b60efc1649c9ba4828d0a098a676 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:09.529258 11051 raft_consensus.cc:2229] T 4325b60efc1649c9ba4828d0a098a676 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:21:09.529812 11051 pending_rounds.cc:62] T 4325b60efc1649c9ba4828d0a098a676 P c8626e300d0644b0908797ad61bd7cef: Trying to abort 1 pending ops.
I20260924 00:21:09.530022 11051 pending_rounds.cc:68] T 4325b60efc1649c9ba4828d0a098a676 P c8626e300d0644b0908797ad61bd7cef: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332697165216419840 op_type: NO_OP noop_request { }
I20260924 00:21:09.530366 11051 raft_consensus.cc:2866] T 4325b60efc1649c9ba4828d0a098a676 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260924 00:21:09.530644 11051 raft_consensus.cc:2258] T 4325b60efc1649c9ba4828d0a098a676 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:09.532596 11051 ts_tablet_manager.cc:1905] T 4325b60efc1649c9ba4828d0a098a676 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:09.534961 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 18abdec265a24dc8bfc765d4395767c0 (table default.Table_a5a83435a7fb47ce8083022c0686b74a [id=a587a50b0c204843a41acacc45c2b150]) successfully deleted
W20260924 00:21:09.535871 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet df24a60e710241de9110afc5a6167bf9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet df24a60e710241de9110afc5a6167bf9 already in progress: creating tablet
I20260924 00:21:09.540218 10634 tablet_service.cc:1467] Processing CreateTablet for tablet f4db89087d404354b4392344715b7e97 (DEFAULT_TABLE table=default.Table_5bdf50c2b69b4f3cadbb52a11cf10035 [id=b07edcb7ee5044c6a486984cc3efe917]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.541124 10634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4db89087d404354b4392344715b7e97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.554833 10915 tablet_service.cc:1467] Processing CreateTablet for tablet f4db89087d404354b4392344715b7e97 (DEFAULT_TABLE table=default.Table_5bdf50c2b69b4f3cadbb52a11cf10035 [id=b07edcb7ee5044c6a486984cc3efe917]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.555843 10915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4db89087d404354b4392344715b7e97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.564332 11041 raft_consensus.cc:348] T 59a0a950a8144e99b2ed47be9a567d9a P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:09.565021 11041 raft_consensus.cc:374] T 59a0a950a8144e99b2ed47be9a567d9a P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:09.565250 11041 raft_consensus.cc:729] T 59a0a950a8144e99b2ed47be9a567d9a P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:09.566085 10912 tablet_service.cc:1467] Processing CreateTablet for tablet 4aa0dc5c502a406b99609f044b6baeb0 (DEFAULT_TABLE table=default.Table_82b79ff8ffca4a7880702b55cd10e393 [id=7e160144b1704e27a1d7f729c7a04049]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.567183 10912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4aa0dc5c502a406b99609f044b6baeb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.571376 10780 tablet_service.cc:1467] Processing CreateTablet for tablet 90d5d35a4a8d4bac99ec458d743a0478 (DEFAULT_TABLE table=default.Table_30b352db3bec45a2ac02e7cea36b0d0c [id=d350b454848949898379a9021ab89fa8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.572255 10780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90d5d35a4a8d4bac99ec458d743a0478. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.573768 11040 raft_consensus.cc:348] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:09.578658 11040 raft_consensus.cc:374] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:09.578904 11040 raft_consensus.cc:729] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:09.579489 11040 consensus_queue.cc:260] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:09.582314 10628 tablet_service.cc:1467] Processing CreateTablet for tablet 90d5d35a4a8d4bac99ec458d743a0478 (DEFAULT_TABLE table=default.Table_30b352db3bec45a2ac02e7cea36b0d0c [id=d350b454848949898379a9021ab89fa8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.583379 10628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90d5d35a4a8d4bac99ec458d743a0478. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.589867 11051 ts_tablet_manager.cc:1918] T 4325b60efc1649c9ba4828d0a098a676 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:09.590281 11051 log.cc:1196] T 4325b60efc1649c9ba4828d0a098a676 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/4325b60efc1649c9ba4828d0a098a676
I20260924 00:21:09.590917 11051 ts_tablet_manager.cc:1939] T 4325b60efc1649c9ba4828d0a098a676 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:09.593742 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 4325b60efc1649c9ba4828d0a098a676 (table default.Table_4a5edca73a094b4d93ba4d389679ab48 [id=7a76983970cd491a8e8635db379bdd10]) successfully deleted
I20260924 00:21:09.599325 11039 tablet_bootstrap.cc:492] T a0f71049f6b3478da3b07ebc9293ce64 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:09.601086 11039 ts_tablet_manager.cc:1397] T a0f71049f6b3478da3b07ebc9293ce64 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.484s user 0.005s sys 0.005s
I20260924 00:21:09.575870 10779 tablet_service.cc:1467] Processing CreateTablet for tablet f4db89087d404354b4392344715b7e97 (DEFAULT_TABLE table=default.Table_5bdf50c2b69b4f3cadbb52a11cf10035 [id=b07edcb7ee5044c6a486984cc3efe917]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.603835 10779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4db89087d404354b4392344715b7e97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.607090 10627 tablet_service.cc:1467] Processing CreateTablet for tablet 57a408ae6e4e439a852381a803bb8acf (DEFAULT_TABLE table=default.Table_99f14ba135cc459883dbb7c85dc21e30 [id=267c2c9545e547199df03ec06dda22be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.610113 10627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57a408ae6e4e439a852381a803bb8acf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.611714 11040 ts_tablet_manager.cc:1428] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.088s user 0.005s sys 0.000s
I20260924 00:21:09.611385 10915 tablet_service.cc:1467] Processing CreateTablet for tablet 90d5d35a4a8d4bac99ec458d743a0478 (DEFAULT_TABLE table=default.Table_30b352db3bec45a2ac02e7cea36b0d0c [id=d350b454848949898379a9021ab89fa8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.615427 10915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90d5d35a4a8d4bac99ec458d743a0478. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.621040 10909 tablet_service.cc:1467] Processing CreateTablet for tablet 57a408ae6e4e439a852381a803bb8acf (DEFAULT_TABLE table=default.Table_99f14ba135cc459883dbb7c85dc21e30 [id=267c2c9545e547199df03ec06dda22be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.621986 10909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57a408ae6e4e439a852381a803bb8acf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.600693 11041 consensus_queue.cc:260] T 59a0a950a8144e99b2ed47be9a567d9a P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:09.620046 10634 tablet_service.cc:1467] Processing CreateTablet for tablet 4c83e8c5109b43fb8268ec3ed52199be (DEFAULT_TABLE table=default.Table_ee28da98c4ae45d28bb5728f390a54b9 [id=bf5691bc3ec148eda2cb116bb7842cfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.625648 10634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c83e8c5109b43fb8268ec3ed52199be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.612386 10780 tablet_service.cc:1467] Processing CreateTablet for tablet 57a408ae6e4e439a852381a803bb8acf (DEFAULT_TABLE table=default.Table_99f14ba135cc459883dbb7c85dc21e30 [id=267c2c9545e547199df03ec06dda22be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.626362 10780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57a408ae6e4e439a852381a803bb8acf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.619684 10777 tablet_service.cc:1467] Processing CreateTablet for tablet 4c83e8c5109b43fb8268ec3ed52199be (DEFAULT_TABLE table=default.Table_ee28da98c4ae45d28bb5728f390a54b9 [id=bf5691bc3ec148eda2cb116bb7842cfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.633359 10777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c83e8c5109b43fb8268ec3ed52199be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.638391 10778 tablet_service.cc:1467] Processing CreateTablet for tablet c467913f46b84705a1718372d90ad579 (DEFAULT_TABLE table=default.Table_35151b4e43bd4e3bbf312c8f1c619e91 [id=3bdd316445814324a47269d8249a2089]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.638849 10625 tablet_service.cc:1467] Processing CreateTablet for tablet c467913f46b84705a1718372d90ad579 (DEFAULT_TABLE table=default.Table_35151b4e43bd4e3bbf312c8f1c619e91 [id=3bdd316445814324a47269d8249a2089]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.639369 10778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c467913f46b84705a1718372d90ad579. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.639755 10625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c467913f46b84705a1718372d90ad579. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.650133 11041 ts_tablet_manager.cc:1428] T 59a0a950a8144e99b2ed47be9a567d9a P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.144s user 0.002s sys 0.004s
I20260924 00:21:09.657960 11040 tablet_bootstrap.cc:492] T 4420cead1f284af8958f189557833098 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:09.657923 10920 tablet_service.cc:1467] Processing CreateTablet for tablet c467913f46b84705a1718372d90ad579 (DEFAULT_TABLE table=default.Table_35151b4e43bd4e3bbf312c8f1c619e91 [id=3bdd316445814324a47269d8249a2089]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.658842 10920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c467913f46b84705a1718372d90ad579. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.665889 10907 tablet_service.cc:1467] Processing CreateTablet for tablet 0856ff2896ce4832af348947f226ee7b (DEFAULT_TABLE table=default.Table_9c2aa717e62340c785be379c0dd5837c [id=737634d9f0f4413c8b6dcaa26de87e2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.666891 10907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0856ff2896ce4832af348947f226ee7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.676116 10366 master_service.cc:946] ReplaceTablet: received request to replace tablet 57ae3d10846a4aaa84eba81b13aafd44 from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:09.659329 10631 tablet_service.cc:1467] Processing CreateTablet for tablet 0856ff2896ce4832af348947f226ee7b (DEFAULT_TABLE table=default.Table_9c2aa717e62340c785be379c0dd5837c [id=737634d9f0f4413c8b6dcaa26de87e2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.677891 10631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0856ff2896ce4832af348947f226ee7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.683863 10374 master_service.cc:946] ReplaceTablet: received request to replace tablet df24a60e710241de9110afc5a6167bf9 from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:09.685631 11041 tablet_bootstrap.cc:492] T 4e9f00abffe14046bf47a6cae7010690 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:09.688265 10365 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_29e523da31634984b775f31084e82c99" } modify_external_catalogs: true
I20260924 00:21:09.688750 10365 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_29e523da31634984b775f31084e82c99" } modify_external_catalogs: true
I20260924 00:21:09.702869 10371 catalog_manager.cc:6040] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: default.Table_07390b277c404c7c9daecd17fe3dccca [id=12285914811f4220822b47a2a62b01ab] alter complete (version 3)
I20260924 00:21:09.705422 10775 tablet_service.cc:1514] Processing DeleteTablet for tablet 18abdec265a24dc8bfc765d4395767c0 with delete_type TABLET_DATA_DELETED (replaced by tablet 45ba52ea61364b988ce77b265f8c7a79) from {username='slave'} at 127.0.0.1:35996
W20260924 00:21:09.706653 10361 catalog_manager.cc:4808] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 18abdec265a24dc8bfc765d4395767c0 because the tablet was not found. No further retry: Not found: Tablet not found: 18abdec265a24dc8bfc765d4395767c0
W20260924 00:21:09.707201 10372 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 69061414550b4bd9bf4b2f40a38d8824 consensus state has no leader
W20260924 00:21:09.708648 10368 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 61ca7994dac34681847fe5748509fe24 has no consensus state
I20260924 00:21:09.709085 10909 tablet_service.cc:1467] Processing CreateTablet for tablet 4c83e8c5109b43fb8268ec3ed52199be (DEFAULT_TABLE table=default.Table_ee28da98c4ae45d28bb5728f390a54b9 [id=bf5691bc3ec148eda2cb116bb7842cfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.710464 10780 tablet_service.cc:1467] Processing CreateTablet for tablet 0856ff2896ce4832af348947f226ee7b (DEFAULT_TABLE table=default.Table_9c2aa717e62340c785be379c0dd5837c [id=737634d9f0f4413c8b6dcaa26de87e2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.711185 10369 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet f77aea0aad5a430f941aaec0a1c19640
I20260924 00:21:09.715948 11040 tablet_bootstrap.cc:654] T 4420cead1f284af8958f189557833098 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:09.718300 10908 tablet_service.cc:1514] Processing DeleteTablet for tablet f77aea0aad5a430f941aaec0a1c19640 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:09 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:09.726334 10366 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 57ae3d10846a4aaa84eba81b13aafd44
I20260924 00:21:09.727800 10774 tablet_service.cc:1514] Processing DeleteTablet for tablet 57ae3d10846a4aaa84eba81b13aafd44 with delete_type TABLET_DATA_DELETED (replaced by tablet b0b3629be7c44bc4b3a79028e8dc33e9) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:09.728710 10908 tablet_service.cc:1514] Processing DeleteTablet for tablet 60ebab3e93f54383b87aeb54bbde722d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:08 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:09.729439 10628 tablet_service.cc:1514] Processing DeleteTablet for tablet f77aea0aad5a430f941aaec0a1c19640 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:09 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:09.730316 11049 tablet_replica.cc:330] T f77aea0aad5a430f941aaec0a1c19640 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
W20260924 00:21:09.731524 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 57ae3d10846a4aaa84eba81b13aafd44 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 57ae3d10846a4aaa84eba81b13aafd44 already in progress: creating tablet
I20260924 00:21:09.734124 11049 raft_consensus.cc:2229] T f77aea0aad5a430f941aaec0a1c19640 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:09.734555 11049 raft_consensus.cc:2258] T f77aea0aad5a430f941aaec0a1c19640 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:09.738214 11041 tablet_bootstrap.cc:654] T 4e9f00abffe14046bf47a6cae7010690 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:09.739214 11039 raft_consensus.cc:348] T a0f71049f6b3478da3b07ebc9293ce64 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:09.739914 11039 raft_consensus.cc:374] T a0f71049f6b3478da3b07ebc9293ce64 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:09.740186 11039 raft_consensus.cc:729] T a0f71049f6b3478da3b07ebc9293ce64 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:09.740818 11039 consensus_queue.cc:260] T a0f71049f6b3478da3b07ebc9293ce64 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:09.742944 10908 tablet_service.cc:1514] Processing DeleteTablet for tablet 57ae3d10846a4aaa84eba81b13aafd44 with delete_type TABLET_DATA_DELETED (replaced by tablet b0b3629be7c44bc4b3a79028e8dc33e9) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:09.743155 11039 ts_tablet_manager.cc:1428] T a0f71049f6b3478da3b07ebc9293ce64 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.123s user 0.005s sys 0.002s
I20260924 00:21:09.744086 10780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0856ff2896ce4832af348947f226ee7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.744123 10366 catalog_manager.cc:6565] ReplaceTablet: tablet 57ae3d10846a4aaa84eba81b13aafd44 deleted and replaced by tablet b0b3629be7c44bc4b3a79028e8dc33e9
I20260924 00:21:09.736127 11049 ts_tablet_manager.cc:1905] T f77aea0aad5a430f941aaec0a1c19640 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:09.752631 10371 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet bf144ab1a6464a4ab02ac0642c5255e3 (table default.Table_382099cf77a24bb8bd6fe084926431c9 [id=3db896d304514ead8dfe726c402a6fcd]). Expected version 1 got 0
I20260924 00:21:09.747517 10634 tablet_service.cc:1514] Processing DeleteTablet for tablet a72742864a6b4299b71757fb308dcadf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:06 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:09.759433 10634 tablet_service.cc:1514] Processing DeleteTablet for tablet 2499754460e04c27ac53d52a4a67f557 with delete_type TABLET_DATA_DELETED (replaced by tablet 7673597f1afc43198e86929a2939269a) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:09.761224 10908 tablet_service.cc:1514] Processing DeleteTablet for tablet 18abdec265a24dc8bfc765d4395767c0 with delete_type TABLET_DATA_DELETED (replaced by tablet 45ba52ea61364b988ce77b265f8c7a79) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:09.762276 10908 tablet_service.cc:1514] Processing DeleteTablet for tablet 95a7a1d2147c4d7ea63ca37a8f0ea4e6 with delete_type TABLET_DATA_DELETED (replaced by tablet 3b8d74761b0b4fd0a22021d3870ec05c) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:09.762907 10634 tablet_service.cc:1514] Processing DeleteTablet for tablet b36cbb6520de4df58b88cfa53c20ee7a with delete_type TABLET_DATA_DELETED (replaced by tablet 9e44244181324288ad04afb7096839cb) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:09.763218 10908 tablet_service.cc:1514] Processing DeleteTablet for tablet 4325b60efc1649c9ba4828d0a098a676 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:08 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:09.764142 10908 tablet_service.cc:1514] Processing DeleteTablet for tablet a72742864a6b4299b71757fb308dcadf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:06 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:09.756000 10775 tablet_service.cc:1514] Processing DeleteTablet for tablet f77aea0aad5a430f941aaec0a1c19640 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:09 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:09.765642 11335 tablet_replica.cc:330] T f77aea0aad5a430f941aaec0a1c19640 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:09.766327 11335 raft_consensus.cc:2229] T f77aea0aad5a430f941aaec0a1c19640 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:09.766674 10774 tablet_service.cc:1514] Processing DeleteTablet for tablet 95a7a1d2147c4d7ea63ca37a8f0ea4e6 with delete_type TABLET_DATA_DELETED (replaced by tablet 3b8d74761b0b4fd0a22021d3870ec05c) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:09.766781 11335 raft_consensus.cc:2258] T f77aea0aad5a430f941aaec0a1c19640 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:09.757799 10365 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet b0b3629be7c44bc4b3a79028e8dc33e9
I20260924 00:21:09.768640 11335 ts_tablet_manager.cc:1905] T f77aea0aad5a430f941aaec0a1c19640 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:09.774036 11051 tablet_replica.cc:330] T f77aea0aad5a430f941aaec0a1c19640 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:09.774587 11039 tablet_bootstrap.cc:492] T 2499754460e04c27ac53d52a4a67f557 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:09.774739 11051 raft_consensus.cc:2229] T f77aea0aad5a430f941aaec0a1c19640 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:21:09.775465 11051 raft_consensus.cc:2258] T f77aea0aad5a430f941aaec0a1c19640 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:09.777031 11051 ts_tablet_manager.cc:1905] T f77aea0aad5a430f941aaec0a1c19640 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260924 00:21:09.777251 10359 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 69061414550b4bd9bf4b2f40a38d8824 (table default.Table_cf2f09bc8e6849848a9e641521c37450 [id=d7d5340352b948e3a72b210b1e99cc25], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 69061414550b4bd9bf4b2f40a38d8824 consensus state has no leader
I20260924 00:21:09.782869 10365 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet bf144ab1a6464a4ab02ac0642c5255e3 (table default.Table_382099cf77a24bb8bd6fe084926431c9 [id=3db896d304514ead8dfe726c402a6fcd]). Expected version 1 got 0
I20260924 00:21:09.783157 11039 tablet_bootstrap.cc:654] T 2499754460e04c27ac53d52a4a67f557 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:09.785493 10634 tablet_service.cc:1514] Processing DeleteTablet for tablet 59a0a950a8144e99b2ed47be9a567d9a with delete_type TABLET_DATA_DELETED (replaced by tablet d4e120acff3e4c30aed813b0b24fbfee) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:09.788693 10909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c83e8c5109b43fb8268ec3ed52199be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.791044 10912 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a9443159a334093b4135364057e0c8b with delete_type TABLET_DATA_DELETED (replaced by tablet a53f06276dbb4d9f97a154a8ceb0278d) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:09.797466 11041 tablet_bootstrap.cc:492] T 4e9f00abffe14046bf47a6cae7010690 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:09.798846 11041 ts_tablet_manager.cc:1397] T 4e9f00abffe14046bf47a6cae7010690 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.113s user 0.005s sys 0.006s
I20260924 00:21:09.801828 10908 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c583d9ba1414f8ea2e605b4b6b076f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:01 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:09.801713 11041 raft_consensus.cc:348] T 4e9f00abffe14046bf47a6cae7010690 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:09.802320 11041 raft_consensus.cc:374] T 4e9f00abffe14046bf47a6cae7010690 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:09.810098 11051 ts_tablet_manager.cc:1918] T f77aea0aad5a430f941aaec0a1c19640 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:09.810469 11051 log.cc:1196] T f77aea0aad5a430f941aaec0a1c19640 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/f77aea0aad5a430f941aaec0a1c19640
I20260924 00:21:09.811096 10775 tablet_service.cc:1467] Processing CreateTablet for tablet 7673597f1afc43198e86929a2939269a (DEFAULT_TABLE table=default.Table_6cf69012192f4f1087e29db70ae8c282 [id=93c1734de58245f8b1d5ab88ec9f0c8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.812296 11051 ts_tablet_manager.cc:1939] T f77aea0aad5a430f941aaec0a1c19640 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:09.812072 10775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7673597f1afc43198e86929a2939269a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.820922 11051 tablet_replica.cc:330] T a72742864a6b4299b71757fb308dcadf P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:09.821640 11051 raft_consensus.cc:2229] T a72742864a6b4299b71757fb308dcadf P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:09.822023 11051 raft_consensus.cc:2258] T a72742864a6b4299b71757fb308dcadf P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:09.823144 10774 tablet_service.cc:1467] Processing CreateTablet for tablet 4298ee4a320b470d89d0d4897bb50d5e (DEFAULT_TABLE table=default.Table_aceb5020c4374a028638d9931d6665c9 [id=100796b0a2be4926b6fca423e2b164fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.823515 11051 ts_tablet_manager.cc:1905] T a72742864a6b4299b71757fb308dcadf P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:09.824069 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4298ee4a320b470d89d0d4897bb50d5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.829346 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet f77aea0aad5a430f941aaec0a1c19640 (table default.Table_87fc5b8981644a5d862af47154aaa22a [id=da26c922964143e789bc49d07d8eb880]) successfully deleted
W20260924 00:21:09.829953 10358 catalog_manager.cc:4808] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 60ebab3e93f54383b87aeb54bbde722d because the tablet was not found. No further retry: Not found: Tablet not found: 60ebab3e93f54383b87aeb54bbde722d
I20260924 00:21:09.830564 10365 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet bf144ab1a6464a4ab02ac0642c5255e3 (table default.Table_382099cf77a24bb8bd6fe084926431c9 [id=3db896d304514ead8dfe726c402a6fcd]). Expected version 1 got 0
W20260924 00:21:09.830739 10358 catalog_manager.cc:4808] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 18abdec265a24dc8bfc765d4395767c0 because the tablet was not found. No further retry: Not found: Tablet not found: 18abdec265a24dc8bfc765d4395767c0
I20260924 00:21:09.830885 10909 tablet_service.cc:1467] Processing CreateTablet for tablet 7673597f1afc43198e86929a2939269a (DEFAULT_TABLE table=default.Table_6cf69012192f4f1087e29db70ae8c282 [id=93c1734de58245f8b1d5ab88ec9f0c8e]), partition=RANGE (key) PARTITION UNBOUNDED
W20260924 00:21:09.831218 10358 catalog_manager.cc:4808] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 95a7a1d2147c4d7ea63ca37a8f0ea4e6 because the tablet was not found. No further retry: Not found: Tablet not found: 95a7a1d2147c4d7ea63ca37a8f0ea4e6
I20260924 00:21:09.831794 10909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7673597f1afc43198e86929a2939269a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:21:09.831782 10358 catalog_manager.cc:4808] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 4325b60efc1649c9ba4828d0a098a676 because the tablet was not found. No further retry: Not found: Tablet not found: 4325b60efc1649c9ba4828d0a098a676
I20260924 00:21:09.803857 10634 tablet_service.cc:1467] Processing CreateTablet for tablet 7673597f1afc43198e86929a2939269a (DEFAULT_TABLE table=default.Table_6cf69012192f4f1087e29db70ae8c282 [id=93c1734de58245f8b1d5ab88ec9f0c8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.809175 10628 tablet_service.cc:1514] Processing DeleteTablet for tablet 57ae3d10846a4aaa84eba81b13aafd44 with delete_type TABLET_DATA_DELETED (replaced by tablet b0b3629be7c44bc4b3a79028e8dc33e9) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:09.833590 10634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7673597f1afc43198e86929a2939269a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.838582 11335 ts_tablet_manager.cc:1918] T f77aea0aad5a430f941aaec0a1c19640 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:09.841020 11041 raft_consensus.cc:729] T 4e9f00abffe14046bf47a6cae7010690 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:09.841619 11041 consensus_queue.cc:260] T 4e9f00abffe14046bf47a6cae7010690 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:09.846946 11040 tablet_bootstrap.cc:492] T 4420cead1f284af8958f189557833098 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:09.847121 11051 ts_tablet_manager.cc:1918] T a72742864a6b4299b71757fb308dcadf P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:09.847334 11040 ts_tablet_manager.cc:1397] T 4420cead1f284af8958f189557833098 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.190s user 0.004s sys 0.004s
I20260924 00:21:09.849421 11040 raft_consensus.cc:348] T 4420cead1f284af8958f189557833098 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:09.850032 11040 raft_consensus.cc:374] T 4420cead1f284af8958f189557833098 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:09.850298 11040 raft_consensus.cc:729] T 4420cead1f284af8958f189557833098 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:09.851815 10371 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 4e9f00abffe14046bf47a6cae7010690 (table default.Table_a23f89d2bec746fda3e52abc5a0cea95 [id=423a3ef82b0f488fa8fb082d4e6f63a3]). Expected version 2 got 0
I20260924 00:21:09.828246 11049 ts_tablet_manager.cc:1918] T f77aea0aad5a430f941aaec0a1c19640 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:09.838934 11335 log.cc:1196] T f77aea0aad5a430f941aaec0a1c19640 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/f77aea0aad5a430f941aaec0a1c19640
I20260924 00:21:09.855343 11049 log.cc:1196] T f77aea0aad5a430f941aaec0a1c19640 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/f77aea0aad5a430f941aaec0a1c19640
I20260924 00:21:09.855883 11335 ts_tablet_manager.cc:1939] T f77aea0aad5a430f941aaec0a1c19640 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:09.856024 11049 ts_tablet_manager.cc:1939] T f77aea0aad5a430f941aaec0a1c19640 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:09.858086 11049 tablet_replica.cc:330] T a72742864a6b4299b71757fb308dcadf P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:09.858115 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet f77aea0aad5a430f941aaec0a1c19640 (table default.Table_87fc5b8981644a5d862af47154aaa22a [id=da26c922964143e789bc49d07d8eb880]) successfully deleted
I20260924 00:21:09.858179 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet f77aea0aad5a430f941aaec0a1c19640 (table default.Table_87fc5b8981644a5d862af47154aaa22a [id=da26c922964143e789bc49d07d8eb880]) successfully deleted
I20260924 00:21:09.858739 11049 raft_consensus.cc:2229] T a72742864a6b4299b71757fb308dcadf P b10a9082a37b46f09d581944914c1670 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:21:09.859324 11049 pending_rounds.cc:62] T a72742864a6b4299b71757fb308dcadf P b10a9082a37b46f09d581944914c1670: Trying to abort 1 pending ops.
I20260924 00:21:09.859529 11049 pending_rounds.cc:68] T a72742864a6b4299b71757fb308dcadf P b10a9082a37b46f09d581944914c1670: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332697166195998720 op_type: NO_OP noop_request { }
I20260924 00:21:09.859855 11049 raft_consensus.cc:2866] T a72742864a6b4299b71757fb308dcadf P b10a9082a37b46f09d581944914c1670 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260924 00:21:09.860147 11049 raft_consensus.cc:2258] T a72742864a6b4299b71757fb308dcadf P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:09.847438 11051 log.cc:1196] T a72742864a6b4299b71757fb308dcadf P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/a72742864a6b4299b71757fb308dcadf
I20260924 00:21:09.861941 11051 ts_tablet_manager.cc:1939] T a72742864a6b4299b71757fb308dcadf P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:09.856884 11041 ts_tablet_manager.cc:1428] T 4e9f00abffe14046bf47a6cae7010690 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.058s user 0.003s sys 0.002s
I20260924 00:21:09.862995 11335 tablet_replica.cc:330] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:09.864076 11051 tablet_replica.cc:330] T 8a9443159a334093b4135364057e0c8b P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:09.864099 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet a72742864a6b4299b71757fb308dcadf (table default.Table_1c64926dc3894391b7b3776711b0c487 [id=8264957c289442e0b31b70cb7e17f7bc]) successfully deleted
I20260924 00:21:09.864712 11051 raft_consensus.cc:2229] T 8a9443159a334093b4135364057e0c8b P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:09.865110 11051 raft_consensus.cc:2258] T 8a9443159a334093b4135364057e0c8b P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:09.865939 11049 ts_tablet_manager.cc:1905] T a72742864a6b4299b71757fb308dcadf P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:09.866134 11039 tablet_bootstrap.cc:492] T 2499754460e04c27ac53d52a4a67f557 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:09.866468 11039 ts_tablet_manager.cc:1397] T 2499754460e04c27ac53d52a4a67f557 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.092s user 0.003s sys 0.005s
I20260924 00:21:09.866526 11051 ts_tablet_manager.cc:1905] T 8a9443159a334093b4135364057e0c8b P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:09.868584 11039 raft_consensus.cc:348] T 2499754460e04c27ac53d52a4a67f557 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:09.870057 11039 raft_consensus.cc:374] T 2499754460e04c27ac53d52a4a67f557 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:09.870319 11039 raft_consensus.cc:729] T 2499754460e04c27ac53d52a4a67f557 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:09.870919 11039 consensus_queue.cc:260] T 2499754460e04c27ac53d52a4a67f557 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:09.863677 11335 raft_consensus.cc:2229] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:09.872694 11039 ts_tablet_manager.cc:1428] T 2499754460e04c27ac53d52a4a67f557 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260924 00:21:09.873718 10909 tablet_service.cc:1467] Processing CreateTablet for tablet 4298ee4a320b470d89d0d4897bb50d5e (DEFAULT_TABLE table=default.Table_aceb5020c4374a028638d9931d6665c9 [id=100796b0a2be4926b6fca423e2b164fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.875593 10628 tablet_service.cc:1467] Processing CreateTablet for tablet 6d39cd33f352433eaf6857a78236aded (DEFAULT_TABLE table=default.Table_b65a4b1360e74260a30a4d0032e3c025 [id=244f1f1c84a14de9950e855b585177fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.875288 10778 tablet_service.cc:1467] Processing CreateTablet for tablet 6d39cd33f352433eaf6857a78236aded (DEFAULT_TABLE table=default.Table_b65a4b1360e74260a30a4d0032e3c025 [id=244f1f1c84a14de9950e855b585177fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.876291 10634 tablet_service.cc:1467] Processing CreateTablet for tablet 4298ee4a320b470d89d0d4897bb50d5e (DEFAULT_TABLE table=default.Table_aceb5020c4374a028638d9931d6665c9 [id=100796b0a2be4926b6fca423e2b164fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.876585 10778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d39cd33f352433eaf6857a78236aded. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.876643 10628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d39cd33f352433eaf6857a78236aded. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.831218 10365 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 5f2a32d5afff458fab79ffa106a98884 (table default.Table_bf61c7623556425688b99c1c373e4879 [id=4f70e833d7f74d91af0faa01ed52f78b]). Expected version 1 got 0
I20260924 00:21:09.880852 10908 tablet_service.cc:1514] Processing DeleteTablet for tablet a0f71049f6b3478da3b07ebc9293ce64 with delete_type TABLET_DATA_DELETED (replaced by tablet d574642ebc6340fcb5e83c36804ee9f8) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:09.881678 10907 tablet_service.cc:1467] Processing CreateTablet for tablet 6d39cd33f352433eaf6857a78236aded (DEFAULT_TABLE table=default.Table_b65a4b1360e74260a30a4d0032e3c025 [id=244f1f1c84a14de9950e855b585177fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.882362 11335 raft_consensus.cc:2258] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:09.888741 11039 tablet_bootstrap.cc:492] T 72d3bb80e1034881b94eb75de5d900c2 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:09.889231 10634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4298ee4a320b470d89d0d4897bb50d5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.892035 10371 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 4e9f00abffe14046bf47a6cae7010690 (table default.Table_a23f89d2bec746fda3e52abc5a0cea95 [id=423a3ef82b0f488fa8fb082d4e6f63a3]). Expected version 2 got 0
I20260924 00:21:09.894320 10908 tablet_service.cc:1514] Processing DeleteTablet for tablet 2499754460e04c27ac53d52a4a67f557 with delete_type TABLET_DATA_DELETED (replaced by tablet 7673597f1afc43198e86929a2939269a) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:09.895480 10367 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_49cf1873c1f4414fa918497b3a48015b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:09.896153 10370 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_9eeed352b33c49649d52ad128cb67203"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:09.897410 10370 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9eeed352b33c49649d52ad128cb67203 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260924 00:21:09.897464 10367 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_49cf1873c1f4414fa918497b3a48015b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:09.901309 10909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4298ee4a320b470d89d0d4897bb50d5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.869199 11040 consensus_queue.cc:260] T 4420cead1f284af8958f189557833098 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:09.903326 10907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d39cd33f352433eaf6857a78236aded. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.904795 11041 tablet_bootstrap.cc:492] T 6bcfd729e9bd4040b3384ab24164e80d P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:09.918090 11457 raft_consensus.cc:482] T 59a0a950a8144e99b2ed47be9a567d9a P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:09.918239 10366 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_2c873f940f8249f9950e78d583d64e36" } new_table_name: "default.Table_eb75a8c5ce6e48b2ba6c7b79990467ef" modify_external_catalogs: true
I20260924 00:21:09.918574 11457 raft_consensus.cc:504] T 59a0a950a8144e99b2ed47be9a567d9a P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:09.919265 10373 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_dca718fc1c1a4e518bbc52483fdd7a09" } new_table_name: "default.Table_5816c202c13e439cbd04eb58f456ebb5" modify_external_catalogs: true
I20260924 00:21:09.922346 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59a0a950a8144e99b2ed47be9a567d9a" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:09.923369 11457 leader_election.cc:288] T 59a0a950a8144e99b2ed47be9a567d9a P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:09.919145 11040 ts_tablet_manager.cc:1428] T 4420cead1f284af8958f189557833098 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.072s user 0.005s sys 0.001s
I20260924 00:21:09.919972 11335 ts_tablet_manager.cc:1905] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:09.925343 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59a0a950a8144e99b2ed47be9a567d9a" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:09.925534 10372 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_c41522845ef54f468c7686bc6b171fe6" } modify_external_catalogs: true
I20260924 00:21:09.925992 10372 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_c41522845ef54f468c7686bc6b171fe6" } modify_external_catalogs: true
W20260924 00:21:09.923872 10589 leader_election.cc:341] T 59a0a950a8144e99b2ed47be9a567d9a P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
W20260924 00:21:09.926810 10592 leader_election.cc:341] T 59a0a950a8144e99b2ed47be9a567d9a P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:09.927151 10592 leader_election.cc:302] T 59a0a950a8144e99b2ed47be9a567d9a P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:09.928035 11457 raft_consensus.cc:2726] T 59a0a950a8144e99b2ed47be9a567d9a P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:09.930327 11040 tablet_bootstrap.cc:492] T 4325b60efc1649c9ba4828d0a098a676 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:09.937088 11039 tablet_bootstrap.cc:654] T 72d3bb80e1034881b94eb75de5d900c2 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:09.940449 10365 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_8823ba2975054860816e34a4fa104be8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:09.944766 11041 tablet_bootstrap.cc:654] T 6bcfd729e9bd4040b3384ab24164e80d P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:09.947948 11040 tablet_bootstrap.cc:654] T 4325b60efc1649c9ba4828d0a098a676 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
W20260924 00:21:09.947896 10365 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8823ba2975054860816e34a4fa104be8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:09.953191 11051 ts_tablet_manager.cc:1918] T 8a9443159a334093b4135364057e0c8b P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:09.953536 11051 log.cc:1196] T 8a9443159a334093b4135364057e0c8b P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/8a9443159a334093b4135364057e0c8b
I20260924 00:21:09.954078 11051 ts_tablet_manager.cc:1939] T 8a9443159a334093b4135364057e0c8b P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:09.955559 11644 raft_consensus.cc:482] T 0c583d9ba1414f8ea2e605b4b6b076f8 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:09.956048 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 8a9443159a334093b4135364057e0c8b (table default.Table_0179ec39d29c4bad9ed0def94394be47 [id=68d73a1a9f1540809f974b9a21f44c1b]) successfully deleted
I20260924 00:21:09.955945 11644 raft_consensus.cc:504] T 0c583d9ba1414f8ea2e605b4b6b076f8 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:09.958036 11644 leader_election.cc:288] T 0c583d9ba1414f8ea2e605b4b6b076f8 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:09.958319 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c583d9ba1414f8ea2e605b4b6b076f8" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:09.959558 10878 leader_election.cc:341] T 0c583d9ba1414f8ea2e605b4b6b076f8 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:09.960713 11051 tablet_replica.cc:330] T 0c583d9ba1414f8ea2e605b4b6b076f8 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:09.961529 11051 raft_consensus.cc:2229] T 0c583d9ba1414f8ea2e605b4b6b076f8 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:09.962810 11051 raft_consensus.cc:2258] T 0c583d9ba1414f8ea2e605b4b6b076f8 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:09.972442 10369 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_3e21dcff4f0b40299e8ddb8ebb7f3c22" } modify_external_catalogs: true
I20260924 00:21:09.972935 10369 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_3e21dcff4f0b40299e8ddb8ebb7f3c22" } modify_external_catalogs: true
I20260924 00:21:09.983103 10780 tablet_service.cc:1514] Processing DeleteTablet for tablet 4420cead1f284af8958f189557833098 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:01 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:09.954676 10371 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_dbe5555ea7824aabb73edf7e94b89b03"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:09.984844 10371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dbe5555ea7824aabb73edf7e94b89b03 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:09.992396 10780 tablet_service.cc:1467] Processing CreateTablet for tablet c1784f15ef0e4aaeb487f1ea9e52e83e (DEFAULT_TABLE table=default.Table_63927aa1872e41da917d6b92c8acb310 [id=485c42c942b74abbb32bea36d25bc184]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:09.993336 10780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1784f15ef0e4aaeb487f1ea9e52e83e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:09.998656 11051 ts_tablet_manager.cc:1905] T 0c583d9ba1414f8ea2e605b4b6b076f8 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:10.003235 10988 heartbeater.cc:347] Connected to a master server at 127.10.4.124:35345
I20260924 00:21:10.006325 10774 tablet_service.cc:1467] Processing CreateTablet for tablet 0cff84bc75aa41aaac797ac183c7dd4a (DEFAULT_TABLE table=default.Table_69f28290116149ff933a5cf08d920b2f [id=933cfdeb149d4d068b47b3c904c58b56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:10.007458 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cff84bc75aa41aaac797ac183c7dd4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:10.008567 10909 tablet_service.cc:1467] Processing CreateTablet for tablet c1784f15ef0e4aaeb487f1ea9e52e83e (DEFAULT_TABLE table=default.Table_63927aa1872e41da917d6b92c8acb310 [id=485c42c942b74abbb32bea36d25bc184]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:10.009691 10778 tablet_service.cc:1467] Processing CreateTablet for tablet 0de944f51bc944459ceb31b254a38429 (DEFAULT_TABLE table=default.Table_4288ed69ffe84f6babc2d310dd022107 [id=de75a360e1ad4b50bc7e55815ca56f20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:10.010664 10778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0de944f51bc944459ceb31b254a38429. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:10.014567 11041 tablet_bootstrap.cc:492] T 6bcfd729e9bd4040b3384ab24164e80d P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:10.014952 11041 ts_tablet_manager.cc:1397] T 6bcfd729e9bd4040b3384ab24164e80d P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.110s user 0.011s sys 0.000s
W20260924 00:21:10.021065 10366 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a53491b1d40143caa73e802304436574 consensus state has no leader
I20260924 00:21:10.029953 11041 raft_consensus.cc:348] T 6bcfd729e9bd4040b3384ab24164e80d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:10.030645 11041 raft_consensus.cc:374] T 6bcfd729e9bd4040b3384ab24164e80d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:10.030874 11041 raft_consensus.cc:729] T 6bcfd729e9bd4040b3384ab24164e80d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:10.031414 11041 consensus_queue.cc:260] T 6bcfd729e9bd4040b3384ab24164e80d P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:10.032063 10909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1784f15ef0e4aaeb487f1ea9e52e83e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:21:10.038813 10373 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4c2a729a802e4abe9ab4f989a40198be consensus state has no leader
I20260924 00:21:10.039853 10372 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 811a6a39194f4fb293bc61da1beacb62
I20260924 00:21:10.043141 10369 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet a62b85ebb2b84ae19cad3e3e2dd07a59
I20260924 00:21:10.044060 10779 tablet_service.cc:1514] Processing DeleteTablet for tablet 811a6a39194f4fb293bc61da1beacb62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:09 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:10.047825 10779 tablet_service.cc:1514] Processing DeleteTablet for tablet a62b85ebb2b84ae19cad3e3e2dd07a59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:09 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:10.048281 10915 tablet_service.cc:1514] Processing DeleteTablet for tablet 811a6a39194f4fb293bc61da1beacb62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:09 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:10.048388 11040 tablet_bootstrap.cc:492] T 4325b60efc1649c9ba4828d0a098a676 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:10.048751 11040 ts_tablet_manager.cc:1397] T 4325b60efc1649c9ba4828d0a098a676 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.119s user 0.004s sys 0.004s
I20260924 00:21:10.050849 11040 raft_consensus.cc:348] T 4325b60efc1649c9ba4828d0a098a676 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:10.051471 11040 raft_consensus.cc:374] T 4325b60efc1649c9ba4828d0a098a676 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:10.051695 11040 raft_consensus.cc:729] T 4325b60efc1649c9ba4828d0a098a676 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:10.052307 11040 consensus_queue.cc:260] T 4325b60efc1649c9ba4828d0a098a676 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:10.054730 11041 ts_tablet_manager.cc:1428] T 6bcfd729e9bd4040b3384ab24164e80d P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.040s user 0.006s sys 0.000s
I20260924 00:21:10.056588 11041 tablet_bootstrap.cc:492] T cb51d5c6460049108cd7e1974821e578 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:10.072647 10779 tablet_service.cc:1514] Processing DeleteTablet for tablet 4325b60efc1649c9ba4828d0a098a676 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:08 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:10.074714 11041 tablet_bootstrap.cc:654] T cb51d5c6460049108cd7e1974821e578 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:10.076138 11335 ts_tablet_manager.cc:1918] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:10.076506 11335 log.cc:1196] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/95a7a1d2147c4d7ea63ca37a8f0ea4e6
I20260924 00:21:10.077085 11335 ts_tablet_manager.cc:1939] T 95a7a1d2147c4d7ea63ca37a8f0ea4e6 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:10.085088 10920 tablet_service.cc:1467] Processing CreateTablet for tablet 0de944f51bc944459ceb31b254a38429 (DEFAULT_TABLE table=default.Table_4288ed69ffe84f6babc2d310dd022107 [id=de75a360e1ad4b50bc7e55815ca56f20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:10.085996 10920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0de944f51bc944459ceb31b254a38429. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:10.087813 10915 tablet_service.cc:1514] Processing DeleteTablet for tablet a62b85ebb2b84ae19cad3e3e2dd07a59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:09 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:10.088979 10908 tablet_service.cc:1467] Processing CreateTablet for tablet 0cff84bc75aa41aaac797ac183c7dd4a (DEFAULT_TABLE table=default.Table_69f28290116149ff933a5cf08d920b2f [id=933cfdeb149d4d068b47b3c904c58b56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:10.089972 10908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cff84bc75aa41aaac797ac183c7dd4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:10.094514 11040 ts_tablet_manager.cc:1428] T 4325b60efc1649c9ba4828d0a098a676 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.045s user 0.005s sys 0.000s
I20260924 00:21:10.095528 11040 tablet_bootstrap.cc:492] T 8a9443159a334093b4135364057e0c8b P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:10.104544 11644 raft_consensus.cc:482] T a0f71049f6b3478da3b07ebc9293ce64 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:10.105036 11644 raft_consensus.cc:504] T a0f71049f6b3478da3b07ebc9293ce64 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:10.107352 11644 leader_election.cc:288] T a0f71049f6b3478da3b07ebc9293ce64 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:10.108299 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 95a7a1d2147c4d7ea63ca37a8f0ea4e6 (table default.Table_0cc93ce7fcfb44089209422666978c62 [id=c0cd199f74494deda9d088345c4c90a3]) successfully deleted
I20260924 00:21:10.118654 10628 tablet_service.cc:1467] Processing CreateTablet for tablet c1784f15ef0e4aaeb487f1ea9e52e83e (DEFAULT_TABLE table=default.Table_63927aa1872e41da917d6b92c8acb310 [id=485c42c942b74abbb32bea36d25bc184]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:10.120386 10625 tablet_service.cc:1467] Processing CreateTablet for tablet 0de944f51bc944459ceb31b254a38429 (DEFAULT_TABLE table=default.Table_4288ed69ffe84f6babc2d310dd022107 [id=de75a360e1ad4b50bc7e55815ca56f20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:10.120638 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0f71049f6b3478da3b07ebc9293ce64" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:10.121531 10626 tablet_service.cc:1514] Processing DeleteTablet for tablet a62b85ebb2b84ae19cad3e3e2dd07a59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:09 UTC) from {username='slave'} at 127.0.0.1:46896
W20260924 00:21:10.121984 10878 leader_election.cc:341] T a0f71049f6b3478da3b07ebc9293ce64 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:10.122428 11335 tablet_replica.cc:330] T 4420cead1f284af8958f189557833098 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:10.123072 11335 raft_consensus.cc:2229] T 4420cead1f284af8958f189557833098 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:10.123445 11335 raft_consensus.cc:2258] T 4420cead1f284af8958f189557833098 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:10.130263 10625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0de944f51bc944459ceb31b254a38429. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:10.136086 10628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1784f15ef0e4aaeb487f1ea9e52e83e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:10.118654 10631 tablet_service.cc:1467] Processing CreateTablet for tablet 0cff84bc75aa41aaac797ac183c7dd4a (DEFAULT_TABLE table=default.Table_69f28290116149ff933a5cf08d920b2f [id=933cfdeb149d4d068b47b3c904c58b56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:10.138484 10631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cff84bc75aa41aaac797ac183c7dd4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:10.142144 11335 ts_tablet_manager.cc:1905] T 4420cead1f284af8958f189557833098 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:10.153761 11040 tablet_bootstrap.cc:654] T 8a9443159a334093b4135364057e0c8b P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
W20260924 00:21:10.154214 11624 lock_manager.cc:586] Waited 1 seconds to obtain row lock on key '<redacted>' (key hash 12737376921701372339) tablet 00000000000000000000000000000000 cur holder 0x7b5800060600
I20260924 00:21:10.165982 10632 tablet_service.cc:1514] Processing DeleteTablet for tablet 811a6a39194f4fb293bc61da1beacb62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:09 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:10.173740 11457 raft_consensus.cc:482] T 4e9f00abffe14046bf47a6cae7010690 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:10.174278 11457 raft_consensus.cc:504] T 4e9f00abffe14046bf47a6cae7010690 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:10.177239 11040 tablet_bootstrap.cc:492] T 8a9443159a334093b4135364057e0c8b P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:10.177259 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e9f00abffe14046bf47a6cae7010690" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:10.177597 11040 ts_tablet_manager.cc:1397] T 8a9443159a334093b4135364057e0c8b P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.082s user 0.007s sys 0.001s
W20260924 00:21:10.178587 10589 leader_election.cc:341] T 4e9f00abffe14046bf47a6cae7010690 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:10.179602 11040 raft_consensus.cc:348] T 8a9443159a334093b4135364057e0c8b P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:10.180214 11040 raft_consensus.cc:374] T 8a9443159a334093b4135364057e0c8b P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:10.180428 11040 raft_consensus.cc:729] T 8a9443159a334093b4135364057e0c8b P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:10.188669 11051 ts_tablet_manager.cc:1918] T 0c583d9ba1414f8ea2e605b4b6b076f8 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:10.190120 11039 tablet_bootstrap.cc:492] T 72d3bb80e1034881b94eb75de5d900c2 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:10.190505 11039 ts_tablet_manager.cc:1397] T 72d3bb80e1034881b94eb75de5d900c2 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.302s user 0.017s sys 0.011s
W20260924 00:21:10.193308 10876 leader_election.cc:341] T 0c583d9ba1414f8ea2e605b4b6b076f8 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: 0c583d9ba1414f8ea2e605b4b6b076f8
I20260924 00:21:10.193650 10876 leader_election.cc:302] T 0c583d9ba1414f8ea2e605b4b6b076f8 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670
W20260924 00:21:10.194103 10876 raft_consensus.cc:2670] T 0c583d9ba1414f8ea2e605b4b6b076f8 P c8626e300d0644b0908797ad61bd7cef: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260924 00:21:10.194525 11051 log.cc:1196] T 0c583d9ba1414f8ea2e605b4b6b076f8 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/0c583d9ba1414f8ea2e605b4b6b076f8
W20260924 00:21:10.194713 10876 leader_election.cc:341] T a0f71049f6b3478da3b07ebc9293ce64 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: a0f71049f6b3478da3b07ebc9293ce64
I20260924 00:21:10.195022 10876 leader_election.cc:302] T a0f71049f6b3478da3b07ebc9293ce64 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670
I20260924 00:21:10.195191 11051 ts_tablet_manager.cc:1939] T 0c583d9ba1414f8ea2e605b4b6b076f8 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:10.195942 11644 raft_consensus.cc:2726] T a0f71049f6b3478da3b07ebc9293ce64 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:10.195986 11040 consensus_queue.cc:260] T 8a9443159a334093b4135364057e0c8b P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:10.197643 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e9f00abffe14046bf47a6cae7010690" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:10.198899 10592 leader_election.cc:341] T 4e9f00abffe14046bf47a6cae7010690 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:10.199237 10592 leader_election.cc:302] T 4e9f00abffe14046bf47a6cae7010690 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:10.197456 11457 leader_election.cc:288] T 4e9f00abffe14046bf47a6cae7010690 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:10.201251 11040 ts_tablet_manager.cc:1428] T 8a9443159a334093b4135364057e0c8b P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.023s user 0.005s sys 0.001s
I20260924 00:21:10.202018 11040 tablet_bootstrap.cc:492] T 2499754460e04c27ac53d52a4a67f557 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:10.201920 11457 raft_consensus.cc:2726] T 4e9f00abffe14046bf47a6cae7010690 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:10.204154 10845 heartbeater.cc:347] Connected to a master server at 127.10.4.124:35345
I20260924 00:21:10.211066 10371 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_5cf852dcccf3498f96d0e5e8364d1a84"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:10.212513 10371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5cf852dcccf3498f96d0e5e8364d1a84 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:10.212502 11644 raft_consensus.cc:482] T 2499754460e04c27ac53d52a4a67f557 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:10.212911 11644 raft_consensus.cc:504] T 2499754460e04c27ac53d52a4a67f557 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:10.214828 11644 leader_election.cc:288] T 2499754460e04c27ac53d52a4a67f557 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:21:10.215699 10370 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 6bcfd729e9bd4040b3384ab24164e80d (table default.Table_9670783a92074b639ecb0d6c374811e5 [id=df2a5a74db65431c9ecb750184f64709]). Expected version 1 got 0
I20260924 00:21:10.216315 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2499754460e04c27ac53d52a4a67f557" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:10.216908 10654 raft_consensus.cc:2454] T 2499754460e04c27ac53d52a4a67f557 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
I20260924 00:21:10.217909 11335 ts_tablet_manager.cc:1918] T 4420cead1f284af8958f189557833098 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:10.217978 10876 leader_election.cc:302] T 2499754460e04c27ac53d52a4a67f557 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:10.218245 11335 log.cc:1196] T 4420cead1f284af8958f189557833098 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/4420cead1f284af8958f189557833098
I20260924 00:21:10.218793 11335 ts_tablet_manager.cc:1939] T 4420cead1f284af8958f189557833098 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:10.218717 11644 raft_consensus.cc:2781] T 2499754460e04c27ac53d52a4a67f557 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:10.219029 11644 raft_consensus.cc:482] T 2499754460e04c27ac53d52a4a67f557 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:10.219328 11644 raft_consensus.cc:3037] T 2499754460e04c27ac53d52a4a67f557 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:10.220850 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 4420cead1f284af8958f189557833098 (table default.Table_bdbcdcde9ac241d5b22b5e8b0da6a9a7 [id=21e1b3346ba346c883fced1e7a1b6cf6]) successfully deleted
I20260924 00:21:10.221616 11051 tablet_replica.cc:330] T a0f71049f6b3478da3b07ebc9293ce64 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:10.222326 11051 raft_consensus.cc:2229] T a0f71049f6b3478da3b07ebc9293ce64 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:10.222216 10794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2499754460e04c27ac53d52a4a67f557" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:10.222697 11051 raft_consensus.cc:2258] T a0f71049f6b3478da3b07ebc9293ce64 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:10.220850 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 0c583d9ba1414f8ea2e605b4b6b076f8 (table default.Table_fd43d6016c81497586729ff4cbe3213c [id=70e3fa6998b649468ec8575758708401]) successfully deleted
I20260924 00:21:10.225219 10369 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_ca083375f0df4255b8654ed88040dfcd" } new_table_name: "default.Table_b7a5eb7bd9fe4e0483e2d47d6182e46d" modify_external_catalogs: true
W20260924 00:21:10.225484 10878 leader_election.cc:341] T 2499754460e04c27ac53d52a4a67f557 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:10.225587 11049 ts_tablet_manager.cc:1918] T a72742864a6b4299b71757fb308dcadf P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:10.192606 11039 raft_consensus.cc:348] T 72d3bb80e1034881b94eb75de5d900c2 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:10.227515 11049 log.cc:1196] T a72742864a6b4299b71757fb308dcadf P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/a72742864a6b4299b71757fb308dcadf
I20260924 00:21:10.227566 11039 raft_consensus.cc:374] T 72d3bb80e1034881b94eb75de5d900c2 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:10.227842 11039 raft_consensus.cc:729] T 72d3bb80e1034881b94eb75de5d900c2 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:10.232064 11637 raft_consensus.cc:482] T 4325b60efc1649c9ba4828d0a098a676 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:10.232465 11637 raft_consensus.cc:504] T 4325b60efc1649c9ba4828d0a098a676 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:10.233309 11039 consensus_queue.cc:260] T 72d3bb80e1034881b94eb75de5d900c2 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:10.234184 11637 leader_election.cc:288] T 4325b60efc1649c9ba4828d0a098a676 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:10.235790 11644 raft_consensus.cc:504] T 2499754460e04c27ac53d52a4a67f557 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
W20260924 00:21:10.238484 10732 leader_election.cc:341] T 4325b60efc1649c9ba4828d0a098a676 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Not found: Tablet not found: 4325b60efc1649c9ba4828d0a098a676
W20260924 00:21:10.238886 10733 leader_election.cc:341] T 4325b60efc1649c9ba4828d0a098a676 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: 4325b60efc1649c9ba4828d0a098a676
I20260924 00:21:10.239241 10733 leader_election.cc:302] T 4325b60efc1649c9ba4828d0a098a676 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08; no voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:10.240247 11637 raft_consensus.cc:2726] T 4325b60efc1649c9ba4828d0a098a676 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:10.242470 10372 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_bf43954da6974fb69d5647190b20cd34"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:10.243872 11041 tablet_bootstrap.cc:492] T cb51d5c6460049108cd7e1974821e578 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
W20260924 00:21:10.243904 10372 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bf43954da6974fb69d5647190b20cd34 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:10.244376 11041 ts_tablet_manager.cc:1397] T cb51d5c6460049108cd7e1974821e578 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.188s user 0.008s sys 0.001s
W20260924 00:21:10.245930 10627 consensus_meta.cc:287] T 57a408ae6e4e439a852381a803bb8acf P b10a9082a37b46f09d581944914c1670: Time spent flushing consensus metadata: real 0.583s user 0.005s sys 0.007s
I20260924 00:21:10.249176 11049 ts_tablet_manager.cc:1939] T a72742864a6b4299b71757fb308dcadf P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:10.252301 11040 tablet_bootstrap.cc:654] T 2499754460e04c27ac53d52a4a67f557 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:10.253671 10373 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_e7d52c1c0aae4f8d841884dc3617ca9c" } new_table_name: "default.Table_7a350a3ca82c427298dd0bedb9db3d60" modify_external_catalogs: true
I20260924 00:21:10.254033 11051 ts_tablet_manager.cc:1905] T a0f71049f6b3478da3b07ebc9293ce64 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:10.256974 10367 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_30b352db3bec45a2ac02e7cea36b0d0c" } modify_external_catalogs: true
I20260924 00:21:10.257467 10367 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_30b352db3bec45a2ac02e7cea36b0d0c" } modify_external_catalogs: true
I20260924 00:21:10.258528 11039 ts_tablet_manager.cc:1428] T 72d3bb80e1034881b94eb75de5d900c2 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.068s user 0.006s sys 0.001s
I20260924 00:21:10.258920 10366 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_69f28290116149ff933a5cf08d920b2f" } modify_external_catalogs: true
I20260924 00:21:10.259372 10366 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_69f28290116149ff933a5cf08d920b2f" } modify_external_catalogs: true
I20260924 00:21:10.261169 10365 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_9e32aedf83114328ab54bc931f2a51a3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:10.262564 10365 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9e32aedf83114328ab54bc931f2a51a3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260924 00:21:10.264866 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 811a6a39194f4fb293bc61da1beacb62 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 811a6a39194f4fb293bc61da1beacb62 already in progress: creating tablet
W20260924 00:21:10.270470 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet a62b85ebb2b84ae19cad3e3e2dd07a59 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a62b85ebb2b84ae19cad3e3e2dd07a59 already in progress: creating tablet
I20260924 00:21:10.275214 11335 tablet_replica.cc:330] T 4325b60efc1649c9ba4828d0a098a676 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:10.276152 11041 raft_consensus.cc:348] T cb51d5c6460049108cd7e1974821e578 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:10.276793 11041 raft_consensus.cc:374] T cb51d5c6460049108cd7e1974821e578 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:10.276930 11335 raft_consensus.cc:2229] T 4325b60efc1649c9ba4828d0a098a676 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:10.277053 11041 raft_consensus.cc:729] T cb51d5c6460049108cd7e1974821e578 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:10.277346 11335 raft_consensus.cc:2258] T 4325b60efc1649c9ba4828d0a098a676 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:10.281872 10368 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_e2da51f4110147169f1bc08241696fae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:10.283330 10368 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e2da51f4110147169f1bc08241696fae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:10.285528 10374 master_service.cc:946] ReplaceTablet: received request to replace tablet b1c2eb7623f847a3a5b6964627ab94df from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:10.286379 11335 ts_tablet_manager.cc:1905] T 4325b60efc1649c9ba4828d0a098a676 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:10.292304 11040 tablet_bootstrap.cc:492] T 2499754460e04c27ac53d52a4a67f557 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:10.292704 11040 ts_tablet_manager.cc:1397] T 2499754460e04c27ac53d52a4a67f557 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.091s user 0.007s sys 0.000s
I20260924 00:21:10.277545 11041 consensus_queue.cc:260] T cb51d5c6460049108cd7e1974821e578 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:10.307633 10778 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a9443159a334093b4135364057e0c8b with delete_type TABLET_DATA_DELETED (replaced by tablet a53f06276dbb4d9f97a154a8ceb0278d) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:10.310493 11041 ts_tablet_manager.cc:1428] T cb51d5c6460049108cd7e1974821e578 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.066s user 0.006s sys 0.000s
I20260924 00:21:10.311359 11041 tablet_bootstrap.cc:492] T 10aa64ed664f40629f5f134052df8f5f P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
W20260924 00:21:10.312525 10373 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a5c560a0b5ce4a1ebb8c0011fe9217bf consensus state has no leader
I20260924 00:21:10.307260 11040 raft_consensus.cc:348] T 2499754460e04c27ac53d52a4a67f557 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:10.321252 11040 raft_consensus.cc:374] T 2499754460e04c27ac53d52a4a67f557 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:10.321537 11040 raft_consensus.cc:729] T 2499754460e04c27ac53d52a4a67f557 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:10.326761 11040 consensus_queue.cc:260] T 2499754460e04c27ac53d52a4a67f557 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:10.332573 11039 tablet_bootstrap.cc:492] T 4e9f00abffe14046bf47a6cae7010690 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:10.338248 11049 tablet_replica.cc:330] T 2499754460e04c27ac53d52a4a67f557 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:10.339090 11049 raft_consensus.cc:2229] T 2499754460e04c27ac53d52a4a67f557 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:10.339512 11049 raft_consensus.cc:2258] T 2499754460e04c27ac53d52a4a67f557 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:10.340914 11051 ts_tablet_manager.cc:1918] T a0f71049f6b3478da3b07ebc9293ce64 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:10.341285 11051 log.cc:1196] T a0f71049f6b3478da3b07ebc9293ce64 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/a0f71049f6b3478da3b07ebc9293ce64
I20260924 00:21:10.341820 11051 ts_tablet_manager.cc:1939] T a0f71049f6b3478da3b07ebc9293ce64 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:10.343034 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet a0f71049f6b3478da3b07ebc9293ce64 (table default.Table_9f2df24361da4f9a9340f6f8673043ea [id=59ec3e94e2f44150994a5ffc1d9fa8f4]) successfully deleted
I20260924 00:21:10.345420 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet a72742864a6b4299b71757fb308dcadf (table default.Table_1c64926dc3894391b7b3776711b0c487 [id=8264957c289442e0b31b70cb7e17f7bc]) successfully deleted
I20260924 00:21:10.347071 11051 tablet_replica.cc:330] T 2499754460e04c27ac53d52a4a67f557 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:10.347777 11051 raft_consensus.cc:2229] T 2499754460e04c27ac53d52a4a67f557 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:10.348182 11051 raft_consensus.cc:2258] T 2499754460e04c27ac53d52a4a67f557 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:10.354499 11040 ts_tablet_manager.cc:1428] T 2499754460e04c27ac53d52a4a67f557 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.062s user 0.004s sys 0.001s
I20260924 00:21:10.357281 11049 ts_tablet_manager.cc:1905] T 2499754460e04c27ac53d52a4a67f557 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:10.365007 10651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2499754460e04c27ac53d52a4a67f557" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:21:10.369815 10778 tablet_service.cc:1514] Processing DeleteTablet for tablet 2499754460e04c27ac53d52a4a67f557 with delete_type TABLET_DATA_DELETED (replaced by tablet 7673597f1afc43198e86929a2939269a) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:10.375492 11040 tablet_bootstrap.cc:492] T 60ebab3e93f54383b87aeb54bbde722d P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
W20260924 00:21:10.366182 10876 leader_election.cc:341] T 2499754460e04c27ac53d52a4a67f557 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Illegal state: Tablet not RUNNING: STOPPED
I20260924 00:21:10.398802 11041 tablet_bootstrap.cc:654] T 10aa64ed664f40629f5f134052df8f5f P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:10.418164 11040 tablet_bootstrap.cc:654] T 60ebab3e93f54383b87aeb54bbde722d P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:10.420120 11644 leader_election.cc:288] T 2499754460e04c27ac53d52a4a67f557 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:21:10.423503 10369 master_service.cc:946] ReplaceTablet: received request to replace tablet a62b85ebb2b84ae19cad3e3e2dd07a59 from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:10.428165 11051 ts_tablet_manager.cc:1905] T 2499754460e04c27ac53d52a4a67f557 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:10.429831 11335 ts_tablet_manager.cc:1918] T 4325b60efc1649c9ba4828d0a098a676 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:10.430179 11335 log.cc:1196] T 4325b60efc1649c9ba4828d0a098a676 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/4325b60efc1649c9ba4828d0a098a676
I20260924 00:21:10.430758 11335 ts_tablet_manager.cc:1939] T 4325b60efc1649c9ba4828d0a098a676 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:10.431993 11335 tablet_replica.cc:330] T 8a9443159a334093b4135364057e0c8b P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:10.432622 11335 raft_consensus.cc:2229] T 8a9443159a334093b4135364057e0c8b P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:10.433022 11335 raft_consensus.cc:2258] T 8a9443159a334093b4135364057e0c8b P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:10.434293 11039 tablet_bootstrap.cc:654] T 4e9f00abffe14046bf47a6cae7010690 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:10.439301 11049 ts_tablet_manager.cc:1918] T 2499754460e04c27ac53d52a4a67f557 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:10.445242 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2499754460e04c27ac53d52a4a67f557" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:21:10.445778 10797 raft_consensus.cc:3037] T 2499754460e04c27ac53d52a4a67f557 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:10.449740 11049 log.cc:1196] T 2499754460e04c27ac53d52a4a67f557 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/2499754460e04c27ac53d52a4a67f557
I20260924 00:21:10.451241 11049 ts_tablet_manager.cc:1939] T 2499754460e04c27ac53d52a4a67f557 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:10.452319 10797 raft_consensus.cc:2454] T 2499754460e04c27ac53d52a4a67f557 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 1.
W20260924 00:21:10.454416 10592 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:35359, user_credentials={real_user=slave}} blocked reactor thread for 255617us
I20260924 00:21:10.455847 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 2499754460e04c27ac53d52a4a67f557 (table default.Table_6cf69012192f4f1087e29db70ae8c282 [id=93c1734de58245f8b1d5ab88ec9f0c8e]) successfully deleted
I20260924 00:21:10.456897 11049 tablet_replica.cc:330] T b36cbb6520de4df58b88cfa53c20ee7a P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:10.457865 10878 leader_election.cc:302] T 2499754460e04c27ac53d52a4a67f557 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters: b10a9082a37b46f09d581944914c1670
W20260924 00:21:10.458433 10878 raft_consensus.cc:2670] T 2499754460e04c27ac53d52a4a67f557 P c8626e300d0644b0908797ad61bd7cef: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260924 00:21:10.459137 11049 raft_consensus.cc:2229] T b36cbb6520de4df58b88cfa53c20ee7a P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:10.461268 11040 tablet_bootstrap.cc:492] T 60ebab3e93f54383b87aeb54bbde722d P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:10.461650 11040 ts_tablet_manager.cc:1397] T 60ebab3e93f54383b87aeb54bbde722d P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.086s user 0.007s sys 0.001s
I20260924 00:21:10.461257 11049 raft_consensus.cc:2258] T b36cbb6520de4df58b88cfa53c20ee7a P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:10.463714 11040 raft_consensus.cc:348] T 60ebab3e93f54383b87aeb54bbde722d P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:10.464326 11040 raft_consensus.cc:374] T 60ebab3e93f54383b87aeb54bbde722d P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:10.464564 11040 raft_consensus.cc:729] T 60ebab3e93f54383b87aeb54bbde722d P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:10.431784 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 4325b60efc1649c9ba4828d0a098a676 (table default.Table_4a5edca73a094b4d93ba4d389679ab48 [id=7a76983970cd491a8e8635db379bdd10]) successfully deleted
I20260924 00:21:10.467875 11049 ts_tablet_manager.cc:1905] T b36cbb6520de4df58b88cfa53c20ee7a P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:10.468241 11673 tablet.cc:1722] T 5f2a32d5afff458fab79ffa106a98884 P 48fbd603abe146c585d15196e1b5be08: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260924 00:21:10.481427 11039 tablet_bootstrap.cc:492] T 4e9f00abffe14046bf47a6cae7010690 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:10.482112 11039 ts_tablet_manager.cc:1397] T 4e9f00abffe14046bf47a6cae7010690 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.150s user 0.010s sys 0.000s
I20260924 00:21:10.491736 11041 tablet_bootstrap.cc:492] T 10aa64ed664f40629f5f134052df8f5f P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:10.492134 11041 ts_tablet_manager.cc:1397] T 10aa64ed664f40629f5f134052df8f5f P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.181s user 0.007s sys 0.001s
I20260924 00:21:10.492115 11335 ts_tablet_manager.cc:1905] T 8a9443159a334093b4135364057e0c8b P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:10.497726 10370 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_cf2f09bc8e6849848a9e641521c37450" } new_table_name: "default.Table_c4f41dcffe2a4074b5178b60ec8e0884" modify_external_catalogs: true
I20260924 00:21:10.503443 11669 tablet.cc:1722] T 5f2a32d5afff458fab79ffa106a98884 P c8626e300d0644b0908797ad61bd7cef: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260924 00:21:10.505299 10625 tablet_service.cc:1514] Processing DeleteTablet for tablet cb51d5c6460049108cd7e1974821e578 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:06 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:10.508860 11040 consensus_queue.cc:260] T 60ebab3e93f54383b87aeb54bbde722d P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
W20260924 00:21:10.518961 10370 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 69061414550b4bd9bf4b2f40a38d8824 consensus state has no leader
I20260924 00:21:10.522403 11040 ts_tablet_manager.cc:1428] T 60ebab3e93f54383b87aeb54bbde722d P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.061s user 0.005s sys 0.000s
I20260924 00:21:10.522437 10370 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_bf43954da6974fb69d5647190b20cd34" } new_table_name: "default.Table_e06918102eb24c628d528f20522dd2a4" modify_external_catalogs: true
I20260924 00:21:10.515475 11039 raft_consensus.cc:348] T 4e9f00abffe14046bf47a6cae7010690 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:10.520607 11673 tablet.cc:1722] T 195b033091b543b58ee08417eae2c0e6 P 48fbd603abe146c585d15196e1b5be08: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260924 00:21:10.525820 11039 raft_consensus.cc:374] T 4e9f00abffe14046bf47a6cae7010690 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:10.526336 11039 raft_consensus.cc:729] T 4e9f00abffe14046bf47a6cae7010690 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:10.502310 11671 tablet.cc:1722] T 5f2a32d5afff458fab79ffa106a98884 P b10a9082a37b46f09d581944914c1670: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260924 00:21:10.527379 11039 consensus_queue.cc:260] T 4e9f00abffe14046bf47a6cae7010690 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:10.530378 11675 tablet.cc:1722] T 195b033091b543b58ee08417eae2c0e6 P c8626e300d0644b0908797ad61bd7cef: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260924 00:21:10.530905 11039 ts_tablet_manager.cc:1428] T 4e9f00abffe14046bf47a6cae7010690 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.048s user 0.007s sys 0.000s
W20260924 00:21:10.543372 10370 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c889f258d75448faafa7bb65e2aafc12 has no consensus state
I20260924 00:21:10.545436 11040 tablet_bootstrap.cc:492] T a72742864a6b4299b71757fb308dcadf P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:10.549894 11039 tablet_bootstrap.cc:492] T 59a0a950a8144e99b2ed47be9a567d9a P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:10.567080 11041 raft_consensus.cc:348] T 10aa64ed664f40629f5f134052df8f5f P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:10.567728 11041 raft_consensus.cc:374] T 10aa64ed664f40629f5f134052df8f5f P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:10.567973 11041 raft_consensus.cc:729] T 10aa64ed664f40629f5f134052df8f5f P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:10.573918 11040 tablet_bootstrap.cc:654] T a72742864a6b4299b71757fb308dcadf P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:10.574824 11678 tablet.cc:1722] T 195b033091b543b58ee08417eae2c0e6 P b10a9082a37b46f09d581944914c1670: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260924 00:21:10.579355 10369 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_cbd6d689c0a149d6a22cc163d3efb580"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:10.579694 10370 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_2ff16b0fac094dc1865b40d6c91bbfdc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:10.586172 11039 tablet_bootstrap.cc:654] T 59a0a950a8144e99b2ed47be9a567d9a P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
W20260924 00:21:10.588299 10370 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2ff16b0fac094dc1865b40d6c91bbfdc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:10.590142 10368 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_8ff1c7565eff465da9ae77704e862894"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:10.591284 10369 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cbd6d689c0a149d6a22cc163d3efb580 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260924 00:21:10.591529 10368 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8ff1c7565eff465da9ae77704e862894 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:10.601749 11041 consensus_queue.cc:260] T 10aa64ed664f40629f5f134052df8f5f P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:10.605023 11335 ts_tablet_manager.cc:1918] T 8a9443159a334093b4135364057e0c8b P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:10.605403 11335 log.cc:1196] T 8a9443159a334093b4135364057e0c8b P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/8a9443159a334093b4135364057e0c8b
I20260924 00:21:10.606001 11335 ts_tablet_manager.cc:1939] T 8a9443159a334093b4135364057e0c8b P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:10.608037 11335 tablet_replica.cc:330] T 2499754460e04c27ac53d52a4a67f557 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:10.610438 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 8a9443159a334093b4135364057e0c8b (table default.Table_0179ec39d29c4bad9ed0def94394be47 [id=68d73a1a9f1540809f974b9a21f44c1b]) successfully deleted
I20260924 00:21:10.612903 10372 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_f1ae1e8ac652481680491d0da3355484"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:10.614547 10372 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f1ae1e8ac652481680491d0da3355484 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:10.626148 11041 ts_tablet_manager.cc:1428] T 10aa64ed664f40629f5f134052df8f5f P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.134s user 0.006s sys 0.001s
I20260924 00:21:10.630270 10368 master_service.cc:946] ReplaceTablet: received request to replace tablet 83d67392af814e43a0fa1bee970820b8 from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:10.608731 11335 raft_consensus.cc:2229] T 2499754460e04c27ac53d52a4a67f557 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:10.633293 11335 raft_consensus.cc:2258] T 2499754460e04c27ac53d52a4a67f557 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:10.636999 11335 ts_tablet_manager.cc:1905] T 2499754460e04c27ac53d52a4a67f557 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:10.645593 11041 tablet_bootstrap.cc:492] T 3b8d74761b0b4fd0a22021d3870ec05c P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:10.655014 11675 tablet.cc:1722] T 119d83562ef546cab496cd7b44fecc15 P c8626e300d0644b0908797ad61bd7cef: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260924 00:21:10.656559 11663 raft_consensus.cc:482] T 72d3bb80e1034881b94eb75de5d900c2 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:10.657104 11663 raft_consensus.cc:504] T 72d3bb80e1034881b94eb75de5d900c2 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:10.659449 11663 leader_election.cc:288] T 72d3bb80e1034881b94eb75de5d900c2 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:10.660718 10798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72d3bb80e1034881b94eb75de5d900c2" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:10.661506 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72d3bb80e1034881b94eb75de5d900c2" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:10.662410 10654 raft_consensus.cc:2454] T 72d3bb80e1034881b94eb75de5d900c2 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
I20260924 00:21:10.663445 10876 leader_election.cc:302] T 72d3bb80e1034881b94eb75de5d900c2 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:10.665406 11644 raft_consensus.cc:2781] T 72d3bb80e1034881b94eb75de5d900c2 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:10.665776 11644 raft_consensus.cc:482] T 72d3bb80e1034881b94eb75de5d900c2 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:10.666071 11644 raft_consensus.cc:3037] T 72d3bb80e1034881b94eb75de5d900c2 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
W20260924 00:21:10.666352 10878 leader_election.cc:341] T 72d3bb80e1034881b94eb75de5d900c2 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:10.683537 11051 ts_tablet_manager.cc:1918] T 2499754460e04c27ac53d52a4a67f557 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:10.683930 11051 log.cc:1196] T 2499754460e04c27ac53d52a4a67f557 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/2499754460e04c27ac53d52a4a67f557
I20260924 00:21:10.684489 11051 ts_tablet_manager.cc:1939] T 2499754460e04c27ac53d52a4a67f557 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:10.687861 11673 tablet.cc:1722] T 119d83562ef546cab496cd7b44fecc15 P 48fbd603abe146c585d15196e1b5be08: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260924 00:21:10.690693 11039 tablet_bootstrap.cc:492] T 59a0a950a8144e99b2ed47be9a567d9a P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:10.691071 11039 ts_tablet_manager.cc:1397] T 59a0a950a8144e99b2ed47be9a567d9a P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.141s user 0.009s sys 0.000s
I20260924 00:21:10.698467 11685 tablet.cc:1722] T 119d83562ef546cab496cd7b44fecc15 P b10a9082a37b46f09d581944914c1670: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260924 00:21:10.703758 11039 raft_consensus.cc:348] T 59a0a950a8144e99b2ed47be9a567d9a P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:10.705631 11039 raft_consensus.cc:374] T 59a0a950a8144e99b2ed47be9a567d9a P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:10.706902 11039 raft_consensus.cc:729] T 59a0a950a8144e99b2ed47be9a567d9a P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:10.707634 11039 consensus_queue.cc:260] T 59a0a950a8144e99b2ed47be9a567d9a P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:10.708259 11040 tablet_bootstrap.cc:492] T a72742864a6b4299b71757fb308dcadf P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:10.708627 11040 ts_tablet_manager.cc:1397] T a72742864a6b4299b71757fb308dcadf P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.163s user 0.018s sys 0.002s
I20260924 00:21:10.703344 11041 tablet_bootstrap.cc:654] T 3b8d74761b0b4fd0a22021d3870ec05c P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:10.715397 11040 raft_consensus.cc:348] T a72742864a6b4299b71757fb308dcadf P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:10.716027 11040 raft_consensus.cc:374] T a72742864a6b4299b71757fb308dcadf P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:10.716257 11040 raft_consensus.cc:729] T a72742864a6b4299b71757fb308dcadf P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:10.716723 11039 ts_tablet_manager.cc:1428] T 59a0a950a8144e99b2ed47be9a567d9a P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.025s user 0.006s sys 0.000s
I20260924 00:21:10.722945 10372 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_ee28da98c4ae45d28bb5728f390a54b9" } new_table_name: "default.Table_83435d6ddd64425390ab1b05337d2081" modify_external_catalogs: true
I20260924 00:21:10.729998 11049 ts_tablet_manager.cc:1918] T b36cbb6520de4df58b88cfa53c20ee7a P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:10.730348 11049 log.cc:1196] T b36cbb6520de4df58b88cfa53c20ee7a P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/b36cbb6520de4df58b88cfa53c20ee7a
I20260924 00:21:10.730907 11049 ts_tablet_manager.cc:1939] T b36cbb6520de4df58b88cfa53c20ee7a P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:10.732054 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet b36cbb6520de4df58b88cfa53c20ee7a (table default.Table_dae0ffc8534047588f6f7cb13a610af5 [id=138be013fbcb4f01a6358d062200fe8f]) successfully deleted
I20260924 00:21:10.732885 11049 tablet_replica.cc:330] T 59a0a950a8144e99b2ed47be9a567d9a P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:10.747520 10369 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_153086de28a74809b8415724f702a78e" } new_table_name: "default.Table_16ca7c156bce492ca79eec1b9881fff6" modify_external_catalogs: true
I20260924 00:21:10.748543 11041 tablet_bootstrap.cc:492] T 3b8d74761b0b4fd0a22021d3870ec05c P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:10.748878 11041 ts_tablet_manager.cc:1397] T 3b8d74761b0b4fd0a22021d3870ec05c P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.103s user 0.004s sys 0.004s
I20260924 00:21:10.749838 11049 raft_consensus.cc:2229] T 59a0a950a8144e99b2ed47be9a567d9a P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:10.750289 11049 raft_consensus.cc:2258] T 59a0a950a8144e99b2ed47be9a567d9a P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:10.750911 11041 raft_consensus.cc:348] T 3b8d74761b0b4fd0a22021d3870ec05c P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:10.752756 11041 raft_consensus.cc:374] T 3b8d74761b0b4fd0a22021d3870ec05c P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260924 00:21:10.757411 10372 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4c83e8c5109b43fb8268ec3ed52199be consensus state has no leader
I20260924 00:21:10.761631 11040 consensus_queue.cc:260] T a72742864a6b4299b71757fb308dcadf P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:10.728736 11644 raft_consensus.cc:504] T 72d3bb80e1034881b94eb75de5d900c2 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
W20260924 00:21:10.764562 10369 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a0668cc6cbc547c5bdc6b62a1b38fcde consensus state has no leader
I20260924 00:21:10.768205 11041 raft_consensus.cc:729] T 3b8d74761b0b4fd0a22021d3870ec05c P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:10.768649 11335 ts_tablet_manager.cc:1918] T 2499754460e04c27ac53d52a4a67f557 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:10.769004 11041 consensus_queue.cc:260] T 3b8d74761b0b4fd0a22021d3870ec05c P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:10.770421 11040 ts_tablet_manager.cc:1428] T a72742864a6b4299b71757fb308dcadf P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.062s user 0.002s sys 0.005s
I20260924 00:21:10.771237 11041 ts_tablet_manager.cc:1428] T 3b8d74761b0b4fd0a22021d3870ec05c P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.022s user 0.005s sys 0.000s
I20260924 00:21:10.772051 11041 tablet_bootstrap.cc:492] T e937cee22326474598cef5c1d09b59c2 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:10.782182 10794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72d3bb80e1034881b94eb75de5d900c2" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
W20260924 00:21:10.783540 10878 leader_election.cc:341] T 72d3bb80e1034881b94eb75de5d900c2 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:10.784687 11644 leader_election.cc:288] T 72d3bb80e1034881b94eb75de5d900c2 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:10.798069 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 2499754460e04c27ac53d52a4a67f557 (table default.Table_6cf69012192f4f1087e29db70ae8c282 [id=93c1734de58245f8b1d5ab88ec9f0c8e]) successfully deleted
I20260924 00:21:10.799655 11335 log.cc:1196] T 2499754460e04c27ac53d52a4a67f557 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/2499754460e04c27ac53d52a4a67f557
I20260924 00:21:10.800323 11335 ts_tablet_manager.cc:1939] T 2499754460e04c27ac53d52a4a67f557 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:10.751894 11049 ts_tablet_manager.cc:1905] T 59a0a950a8144e99b2ed47be9a567d9a P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:10.786233 10651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72d3bb80e1034881b94eb75de5d900c2" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:21:10.801230 10651 raft_consensus.cc:3037] T 72d3bb80e1034881b94eb75de5d900c2 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:10.807606 10651 raft_consensus.cc:2454] T 72d3bb80e1034881b94eb75de5d900c2 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 1.
I20260924 00:21:10.792634 10701 heartbeater.cc:347] Connected to a master server at 127.10.4.124:35345
W20260924 00:21:10.809556 10359 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet a0668cc6cbc547c5bdc6b62a1b38fcde (table default.Table_16ca7c156bce492ca79eec1b9881fff6 [id=222d2010881c4b5c9635aa3c28adc72b], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet a0668cc6cbc547c5bdc6b62a1b38fcde consensus state has no leader
I20260924 00:21:10.810842 11040 tablet_bootstrap.cc:492] T 0c583d9ba1414f8ea2e605b4b6b076f8 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:10.810659 11041 tablet_bootstrap.cc:654] T e937cee22326474598cef5c1d09b59c2 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:10.823247 11637 raft_consensus.cc:482] T 60ebab3e93f54383b87aeb54bbde722d P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:10.823695 11637 raft_consensus.cc:504] T 60ebab3e93f54383b87aeb54bbde722d P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:10.825527 11637 leader_election.cc:288] T 60ebab3e93f54383b87aeb54bbde722d P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:10.825223 10369 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_d02a9cefdc804218a76ef33174772e09"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:10.826567 10369 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d02a9cefdc804218a76ef33174772e09 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:10.833482 10372 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_e33891e1d1b745819f1114e54bded8bb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:10.834874 10372 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e33891e1d1b745819f1114e54bded8bb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260924 00:21:10.835172 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 811a6a39194f4fb293bc61da1beacb62 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 811a6a39194f4fb293bc61da1beacb62 already in progress: creating tablet
I20260924 00:21:10.789446 11039 tablet_bootstrap.cc:492] T b36cbb6520de4df58b88cfa53c20ee7a P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:10.838354 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 2499754460e04c27ac53d52a4a67f557 (table default.Table_6cf69012192f4f1087e29db70ae8c282 [id=93c1734de58245f8b1d5ab88ec9f0c8e]) successfully deleted
I20260924 00:21:10.808665 10876 leader_election.cc:302] T 72d3bb80e1034881b94eb75de5d900c2 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08
I20260924 00:21:10.845152 11040 tablet_bootstrap.cc:654] T 0c583d9ba1414f8ea2e605b4b6b076f8 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
W20260924 00:21:10.845911 10733 leader_election.cc:341] T 60ebab3e93f54383b87aeb54bbde722d P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: 60ebab3e93f54383b87aeb54bbde722d
W20260924 00:21:10.847429 10732 leader_election.cc:341] T 60ebab3e93f54383b87aeb54bbde722d P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Not found: Tablet not found: 60ebab3e93f54383b87aeb54bbde722d
I20260924 00:21:10.847795 10732 leader_election.cc:302] T 60ebab3e93f54383b87aeb54bbde722d P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08; no voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:10.848803 11668 raft_consensus.cc:2781] T 72d3bb80e1034881b94eb75de5d900c2 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:10.849223 11668 raft_consensus.cc:686] T 72d3bb80e1034881b94eb75de5d900c2 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Becoming Leader. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Running, Role: LEADER
I20260924 00:21:10.849931 11668 consensus_queue.cc:237] T 72d3bb80e1034881b94eb75de5d900c2 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:10.850507 10370 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_acfad28894184501be9e3f9e38fea952"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:10.851941 10370 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_acfad28894184501be9e3f9e38fea952 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:10.852674 11637 raft_consensus.cc:2726] T 60ebab3e93f54383b87aeb54bbde722d P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:10.853551 10374 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_9e4907fea442455b9ac9c793dfabf496"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:10.854907 10374 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9e4907fea442455b9ac9c793dfabf496 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260924 00:21:10.856390 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet a62b85ebb2b84ae19cad3e3e2dd07a59 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a62b85ebb2b84ae19cad3e3e2dd07a59 already in progress: creating tablet
I20260924 00:21:10.859817 11039 tablet_bootstrap.cc:654] T b36cbb6520de4df58b88cfa53c20ee7a P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:10.878716 11041 tablet_bootstrap.cc:492] T e937cee22326474598cef5c1d09b59c2 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:10.879115 11041 ts_tablet_manager.cc:1397] T e937cee22326474598cef5c1d09b59c2 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.107s user 0.006s sys 0.003s
I20260924 00:21:10.907524 10373 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 72d3bb80e1034881b94eb75de5d900c2 (table default.Table_10ca22fe85d94c4b836dc22d24343797 [id=ca942906de0f4019bbdf72ec886fe6fb]). Expected version 2 got 0
I20260924 00:21:10.908408 10373 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 5f2a32d5afff458fab79ffa106a98884 (table default.Table_bf61c7623556425688b99c1c373e4879 [id=4f70e833d7f74d91af0faa01ed52f78b]). Expected version 1 got 0
I20260924 00:21:10.908424 11657 raft_consensus.cc:482] T 3b8d74761b0b4fd0a22021d3870ec05c P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:10.908895 11039 tablet_bootstrap.cc:492] T b36cbb6520de4df58b88cfa53c20ee7a P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:10.909054 10373 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 195b033091b543b58ee08417eae2c0e6 (table default.Table_b7a5eb7bd9fe4e0483e2d47d6182e46d [id=f79ca8d11aa04d5f8f72d58b798ed4ea]). Expected version 2 got 1
I20260924 00:21:10.908810 11657 raft_consensus.cc:504] T 3b8d74761b0b4fd0a22021d3870ec05c P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:10.909278 11039 ts_tablet_manager.cc:1397] T b36cbb6520de4df58b88cfa53c20ee7a P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.120s user 0.009s sys 0.001s
I20260924 00:21:10.909603 10373 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 119d83562ef546cab496cd7b44fecc15 (table default.Table_76ffb857cf034b6f80598edd90dad953 [id=5fb5e1e90c9440f385e7b04520bcba9c]). Expected version 1 got 0
I20260924 00:21:10.910671 11657 leader_election.cc:288] T 3b8d74761b0b4fd0a22021d3870ec05c P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:10.911550 11039 raft_consensus.cc:348] T b36cbb6520de4df58b88cfa53c20ee7a P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:10.912066 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b8d74761b0b4fd0a22021d3870ec05c" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:10.912281 11039 raft_consensus.cc:374] T b36cbb6520de4df58b88cfa53c20ee7a P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:10.912509 11039 raft_consensus.cc:729] T b36cbb6520de4df58b88cfa53c20ee7a P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
W20260924 00:21:10.913275 10589 leader_election.cc:341] T 3b8d74761b0b4fd0a22021d3870ec05c P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:10.913118 11039 consensus_queue.cc:260] T b36cbb6520de4df58b88cfa53c20ee7a P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:10.914683 10794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b8d74761b0b4fd0a22021d3870ec05c" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:10.915915 10592 leader_election.cc:341] T 3b8d74761b0b4fd0a22021d3870ec05c P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:10.916263 10592 leader_election.cc:302] T 3b8d74761b0b4fd0a22021d3870ec05c P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:10.917649 11657 raft_consensus.cc:2726] T 3b8d74761b0b4fd0a22021d3870ec05c P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:10.924348 11041 raft_consensus.cc:348] T e937cee22326474598cef5c1d09b59c2 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:10.928323 11041 raft_consensus.cc:374] T e937cee22326474598cef5c1d09b59c2 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:10.928578 11041 raft_consensus.cc:729] T e937cee22326474598cef5c1d09b59c2 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:10.929137 11041 consensus_queue.cc:260] T e937cee22326474598cef5c1d09b59c2 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:10.930593 10778 tablet_service.cc:1467] Processing CreateTablet for tablet 52c46283934a4c2180f15c7fc38b91a9 (DEFAULT_TABLE table=default.Table_49cf1873c1f4414fa918497b3a48015b [id=5816a524b40d48629911e69babd7a80a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:10.931506 10778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52c46283934a4c2180f15c7fc38b91a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:10.937527 11039 ts_tablet_manager.cc:1428] T b36cbb6520de4df58b88cfa53c20ee7a P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.028s user 0.005s sys 0.000s
I20260924 00:21:10.944272 10374 master_service.cc:946] ReplaceTablet: received request to replace tablet a6e4030e824047ca9cc92d98707d3335 from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:10.944746 10908 tablet_service.cc:1467] Processing CreateTablet for tablet 52c46283934a4c2180f15c7fc38b91a9 (DEFAULT_TABLE table=default.Table_49cf1873c1f4414fa918497b3a48015b [id=5816a524b40d48629911e69babd7a80a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:10.946165 10908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52c46283934a4c2180f15c7fc38b91a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:10.948853 10625 tablet_service.cc:1467] Processing CreateTablet for tablet 52c46283934a4c2180f15c7fc38b91a9 (DEFAULT_TABLE table=default.Table_49cf1873c1f4414fa918497b3a48015b [id=5816a524b40d48629911e69babd7a80a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:10.949962 10625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52c46283934a4c2180f15c7fc38b91a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:10.954391 10774 tablet_service.cc:1467] Processing CreateTablet for tablet e26b96e7c0ab4523b9a725743e2d7625 (DEFAULT_TABLE table=default.Table_14437b13dbe34d5591c2a180c12c23b1 [id=aa1298d4fa2d40ac93e7c74077f294ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:10.955339 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e26b96e7c0ab4523b9a725743e2d7625. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:10.955613 10920 tablet_service.cc:1467] Processing CreateTablet for tablet e26b96e7c0ab4523b9a725743e2d7625 (DEFAULT_TABLE table=default.Table_14437b13dbe34d5591c2a180c12c23b1 [id=aa1298d4fa2d40ac93e7c74077f294ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:10.956511 10920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e26b96e7c0ab4523b9a725743e2d7625. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:10.956511 10631 tablet_service.cc:1467] Processing CreateTablet for tablet e26b96e7c0ab4523b9a725743e2d7625 (DEFAULT_TABLE table=default.Table_14437b13dbe34d5591c2a180c12c23b1 [id=aa1298d4fa2d40ac93e7c74077f294ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:10.957419 10631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e26b96e7c0ab4523b9a725743e2d7625. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:10.961977 10365 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_aac740991490482b9f371d9cb2076e76" } new_table_name: "default.Table_ceb867d167eb4df8bb2653ede8e37644" modify_external_catalogs: true
I20260924 00:21:10.969667 10366 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 0cff84bc75aa41aaac797ac183c7dd4a
I20260924 00:21:10.972190 10909 tablet_service.cc:1514] Processing DeleteTablet for tablet 0cff84bc75aa41aaac797ac183c7dd4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:10 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:10.972661 10780 tablet_service.cc:1514] Processing DeleteTablet for tablet 0cff84bc75aa41aaac797ac183c7dd4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:10 UTC) from {username='slave'} at 127.0.0.1:35996
W20260924 00:21:10.973678 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 0cff84bc75aa41aaac797ac183c7dd4a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0cff84bc75aa41aaac797ac183c7dd4a already in progress: creating tablet
W20260924 00:21:10.974051 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 0cff84bc75aa41aaac797ac183c7dd4a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0cff84bc75aa41aaac797ac183c7dd4a already in progress: creating tablet
I20260924 00:21:10.976689 10628 tablet_service.cc:1514] Processing DeleteTablet for tablet 0cff84bc75aa41aaac797ac183c7dd4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:10 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:10.981729 11039 tablet_bootstrap.cc:492] T 6bcfd729e9bd4040b3384ab24164e80d P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:10.984714 10366 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_a743710df3db4dc7af777801a59305e6" } new_table_name: "default.Table_9194ac2d0ed7431385925285b2e7fc5d" modify_external_catalogs: true
I20260924 00:21:10.989135 11041 ts_tablet_manager.cc:1428] T e937cee22326474598cef5c1d09b59c2 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.110s user 0.004s sys 0.003s
I20260924 00:21:10.990095 11041 tablet_bootstrap.cc:492] T 675b9e33edcf4d4ea7fa428b476e3945 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:10.991238 10780 tablet_service.cc:1467] Processing CreateTablet for tablet 83c1ce21578341ad8ee28ec44c518dc3 (DEFAULT_TABLE table=default.Table_a214c5e4b9584a1fba04c74e906fe4d3 [id=28864c6aa1f143bfb7b49d98bff16f8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:10.992406 10780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83c1ce21578341ad8ee28ec44c518dc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.008351 11637 raft_consensus.cc:482] T a72742864a6b4299b71757fb308dcadf P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:11.008787 11637 raft_consensus.cc:504] T a72742864a6b4299b71757fb308dcadf P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:11.009321 11657 raft_consensus.cc:482] T e937cee22326474598cef5c1d09b59c2 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:11.009709 11657 raft_consensus.cc:504] T e937cee22326474598cef5c1d09b59c2 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:11.011175 11637 leader_election.cc:288] T a72742864a6b4299b71757fb308dcadf P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:11.011451 11657 leader_election.cc:288] T e937cee22326474598cef5c1d09b59c2 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
W20260924 00:21:11.013266 10733 leader_election.cc:341] T a72742864a6b4299b71757fb308dcadf P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: a72742864a6b4299b71757fb308dcadf
W20260924 00:21:11.013902 10732 leader_election.cc:341] T a72742864a6b4299b71757fb308dcadf P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Not found: Tablet not found: a72742864a6b4299b71757fb308dcadf
I20260924 00:21:11.014130 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e937cee22326474598cef5c1d09b59c2" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:11.014286 10732 leader_election.cc:302] T a72742864a6b4299b71757fb308dcadf P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08; no voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef
W20260924 00:21:11.015285 10589 leader_election.cc:341] T e937cee22326474598cef5c1d09b59c2 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:11.015849 11637 raft_consensus.cc:2726] T a72742864a6b4299b71757fb308dcadf P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:11.016366 10794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e937cee22326474598cef5c1d09b59c2" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:11.017635 10592 leader_election.cc:341] T e937cee22326474598cef5c1d09b59c2 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:11.018020 10592 leader_election.cc:302] T e937cee22326474598cef5c1d09b59c2 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:11.021986 10909 tablet_service.cc:1467] Processing CreateTablet for tablet 83c1ce21578341ad8ee28ec44c518dc3 (DEFAULT_TABLE table=default.Table_a214c5e4b9584a1fba04c74e906fe4d3 [id=28864c6aa1f143bfb7b49d98bff16f8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.025784 10909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83c1ce21578341ad8ee28ec44c518dc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.026189 11039 tablet_bootstrap.cc:654] T 6bcfd729e9bd4040b3384ab24164e80d P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:11.031035 11049 ts_tablet_manager.cc:1918] T 59a0a950a8144e99b2ed47be9a567d9a P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:11.031386 11049 log.cc:1196] T 59a0a950a8144e99b2ed47be9a567d9a P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/59a0a950a8144e99b2ed47be9a567d9a
I20260924 00:21:11.031931 11049 ts_tablet_manager.cc:1939] T 59a0a950a8144e99b2ed47be9a567d9a P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:11.019017 11657 raft_consensus.cc:2726] T e937cee22326474598cef5c1d09b59c2 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:11.039496 10371 catalog_manager.cc:6040] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: default.Table_bf61c7623556425688b99c1c373e4879 [id=4f70e833d7f74d91af0faa01ed52f78b] alter complete (version 1)
I20260924 00:21:11.035702 11657 raft_consensus.cc:482] T 6bcfd729e9bd4040b3384ab24164e80d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:11.045367 11657 raft_consensus.cc:504] T 6bcfd729e9bd4040b3384ab24164e80d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:11.047086 11657 leader_election.cc:288] T 6bcfd729e9bd4040b3384ab24164e80d P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:11.048354 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bcfd729e9bd4040b3384ab24164e80d" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
W20260924 00:21:11.049608 10589 leader_election.cc:341] T 6bcfd729e9bd4040b3384ab24164e80d P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:11.055742 11041 tablet_bootstrap.cc:654] T 675b9e33edcf4d4ea7fa428b476e3945 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:11.057195 11049 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 263) took 1272 ms. Trace:
I20260924 00:21:11.057619 11049 rpcz_store.cc:276] 0924 00:21:09.785238 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:09.785359 (+ 121us) service_pool.cc:227] Handling call
0924 00:21:10.732736 (+947377us) ts_tablet_manager.cc:1145] Deleting tablet 59a0a950a8144e99b2ed47be9a567d9a
0924 00:21:10.732794 (+ 58us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:10.899351 (+166557us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:11.030827 (+131476us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:11.057104 (+ 26277us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":946863}
I20260924 00:21:11.059413 11049 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 259) took 1313 ms. Trace:
I20260924 00:21:11.059717 11049 rpcz_store.cc:276] 0924 00:21:09.745607 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:09.809042 (+ 63435us) service_pool.cc:227] Handling call
0924 00:21:11.059063 (+1250021us) ts_tablet_manager.cc:1145] Deleting tablet 57ae3d10846a4aaa84eba81b13aafd44
0924 00:21:11.059120 (+ 57us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:11.059361 (+ 241us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1225874}
I20260924 00:21:11.062587 11693 raft_consensus.cc:482] T 10aa64ed664f40629f5f134052df8f5f P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:11.065955 11049 tablet_replica.cc:330] T cb51d5c6460049108cd7e1974821e578 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:11.062999 11693 raft_consensus.cc:504] T 10aa64ed664f40629f5f134052df8f5f P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:11.069365 10794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bcfd729e9bd4040b3384ab24164e80d" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:11.071285 10798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10aa64ed664f40629f5f134052df8f5f" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:11.075155 11039 tablet_bootstrap.cc:492] T 6bcfd729e9bd4040b3384ab24164e80d P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:11.075551 11039 ts_tablet_manager.cc:1397] T 6bcfd729e9bd4040b3384ab24164e80d P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.094s user 0.007s sys 0.000s
I20260924 00:21:11.078029 11039 raft_consensus.cc:348] T 6bcfd729e9bd4040b3384ab24164e80d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:11.078441 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 59a0a950a8144e99b2ed47be9a567d9a (table default.Table_faaa8b08d77d4384a41b2008c9cc98ea [id=3fce8b5a6e82492b8a2355183e566142]) successfully deleted
I20260924 00:21:11.078626 11039 raft_consensus.cc:374] T 6bcfd729e9bd4040b3384ab24164e80d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:11.078948 11039 raft_consensus.cc:729] T 6bcfd729e9bd4040b3384ab24164e80d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
W20260924 00:21:11.079407 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet a62b85ebb2b84ae19cad3e3e2dd07a59 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a62b85ebb2b84ae19cad3e3e2dd07a59 already in progress: creating tablet
I20260924 00:21:11.079614 11039 consensus_queue.cc:260] T 6bcfd729e9bd4040b3384ab24164e80d P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
W20260924 00:21:11.079946 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 811a6a39194f4fb293bc61da1beacb62 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 811a6a39194f4fb293bc61da1beacb62 already in progress: creating tablet
W20260924 00:21:11.080721 10592 leader_election.cc:341] T 6bcfd729e9bd4040b3384ab24164e80d P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:11.081099 10592 leader_election.cc:302] T 6bcfd729e9bd4040b3384ab24164e80d P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:11.081697 11657 raft_consensus.cc:2726] T 6bcfd729e9bd4040b3384ab24164e80d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:11.081456 11039 ts_tablet_manager.cc:1428] T 6bcfd729e9bd4040b3384ab24164e80d P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260924 00:21:11.082010 10592 leader_election.cc:341] T 10aa64ed664f40629f5f134052df8f5f P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:11.083071 11049 raft_consensus.cc:2229] T cb51d5c6460049108cd7e1974821e578 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:11.083482 11049 raft_consensus.cc:2258] T cb51d5c6460049108cd7e1974821e578 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:11.089264 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10aa64ed664f40629f5f134052df8f5f" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
W20260924 00:21:11.090441 10589 leader_election.cc:341] T 10aa64ed664f40629f5f134052df8f5f P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:11.088428 10371 catalog_manager.cc:5495] T 72d3bb80e1034881b94eb75de5d900c2 P c8626e300d0644b0908797ad61bd7cef reported cstate change: term changed from 0 to 1, leader changed from <none> to c8626e300d0644b0908797ad61bd7cef (127.10.4.67). New cstate: current_term: 1 leader_uuid: "c8626e300d0644b0908797ad61bd7cef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } health_report { overall_health: UNKNOWN } } }
I20260924 00:21:11.090798 10589 leader_election.cc:302] T 10aa64ed664f40629f5f134052df8f5f P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:11.091657 11657 raft_consensus.cc:2726] T 10aa64ed664f40629f5f134052df8f5f P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:11.091151 10371 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 72d3bb80e1034881b94eb75de5d900c2 (table default.Table_10ca22fe85d94c4b836dc22d24343797 [id=ca942906de0f4019bbdf72ec886fe6fb]). Expected version 2 got 0
I20260924 00:21:11.092962 11693 leader_election.cc:288] T 10aa64ed664f40629f5f134052df8f5f P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:11.093660 10369 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_aa256627fd99496e988fab956cde2057"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:11.095057 10369 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aa256627fd99496e988fab956cde2057 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:11.096446 10631 tablet_service.cc:1467] Processing CreateTablet for tablet 5cb77a79c52f40b7b7b70366dcf6990a (DEFAULT_TABLE table=default.Table_69005ec88288442691befac5eff2adc7 [id=8308fedce3544a18912f343a3f4b519c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.097978 10774 tablet_service.cc:1467] Processing CreateTablet for tablet 5cb77a79c52f40b7b7b70366dcf6990a (DEFAULT_TABLE table=default.Table_69005ec88288442691befac5eff2adc7 [id=8308fedce3544a18912f343a3f4b519c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.098575 11049 ts_tablet_manager.cc:1905] T cb51d5c6460049108cd7e1974821e578 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:11.098891 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5cb77a79c52f40b7b7b70366dcf6990a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.101760 10631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5cb77a79c52f40b7b7b70366dcf6990a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.101974 11040 tablet_bootstrap.cc:492] T 0c583d9ba1414f8ea2e605b4b6b076f8 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:11.102090 10909 tablet_service.cc:1467] Processing CreateTablet for tablet 5cb77a79c52f40b7b7b70366dcf6990a (DEFAULT_TABLE table=default.Table_69005ec88288442691befac5eff2adc7 [id=8308fedce3544a18912f343a3f4b519c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.102377 10775 tablet_service.cc:1467] Processing CreateTablet for tablet ba81ed510bca46fda6935905fa0c29fb (DEFAULT_TABLE table=default.Table_5d4b70d8740f435bb43cb05808ec557e [id=dccf8774891e46649570c04d4ad36e6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.103217 10775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba81ed510bca46fda6935905fa0c29fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.106492 10908 tablet_service.cc:1467] Processing CreateTablet for tablet ba81ed510bca46fda6935905fa0c29fb (DEFAULT_TABLE table=default.Table_5d4b70d8740f435bb43cb05808ec557e [id=dccf8774891e46649570c04d4ad36e6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.102337 11040 ts_tablet_manager.cc:1397] T 0c583d9ba1414f8ea2e605b4b6b076f8 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.292s user 0.005s sys 0.017s
I20260924 00:21:11.111560 11040 raft_consensus.cc:348] T 0c583d9ba1414f8ea2e605b4b6b076f8 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:11.112187 11040 raft_consensus.cc:374] T 0c583d9ba1414f8ea2e605b4b6b076f8 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:11.112426 11040 raft_consensus.cc:729] T 0c583d9ba1414f8ea2e605b4b6b076f8 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:11.113893 10370 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_3245e93d2b494073b531fd16a48f1244"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:11.115296 10370 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3245e93d2b494073b531fd16a48f1244 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:11.117650 10909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5cb77a79c52f40b7b7b70366dcf6990a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.118162 11040 consensus_queue.cc:260] T 0c583d9ba1414f8ea2e605b4b6b076f8 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:11.118616 11039 tablet_bootstrap.cc:492] T cb51d5c6460049108cd7e1974821e578 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:11.120244 10908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba81ed510bca46fda6935905fa0c29fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.121938 11644 raft_consensus.cc:482] T 4e9f00abffe14046bf47a6cae7010690 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:11.122316 11644 raft_consensus.cc:504] T 4e9f00abffe14046bf47a6cae7010690 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:11.124539 11644 leader_election.cc:288] T 4e9f00abffe14046bf47a6cae7010690 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:11.125164 10371 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 4e9f00abffe14046bf47a6cae7010690 (table default.Table_a23f89d2bec746fda3e52abc5a0cea95 [id=423a3ef82b0f488fa8fb082d4e6f63a3]). Expected version 2 got 0
I20260924 00:21:11.125989 10651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e9f00abffe14046bf47a6cae7010690" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:11.126605 10651 raft_consensus.cc:2454] T 4e9f00abffe14046bf47a6cae7010690 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
I20260924 00:21:11.127797 10876 leader_election.cc:302] T 4e9f00abffe14046bf47a6cae7010690 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:11.129017 10368 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 83d67392af814e43a0fa1bee970820b8
I20260924 00:21:11.130214 10368 catalog_manager.cc:6565] ReplaceTablet: tablet 83d67392af814e43a0fa1bee970820b8 deleted and replaced by tablet 39cdf419c8b24274bbad1fe4444dd73e
I20260924 00:21:11.132591 11644 raft_consensus.cc:2781] T 4e9f00abffe14046bf47a6cae7010690 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:11.132874 11644 raft_consensus.cc:482] T 4e9f00abffe14046bf47a6cae7010690 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:11.133172 11644 raft_consensus.cc:3037] T 4e9f00abffe14046bf47a6cae7010690 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:11.136053 10920 tablet_service.cc:1514] Processing DeleteTablet for tablet 83d67392af814e43a0fa1bee970820b8 with delete_type TABLET_DATA_DELETED (replaced by tablet 39cdf419c8b24274bbad1fe4444dd73e) from {username='slave'} at 127.0.0.1:35358
W20260924 00:21:11.137251 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 83d67392af814e43a0fa1bee970820b8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 83d67392af814e43a0fa1bee970820b8 already in progress: creating tablet
I20260924 00:21:11.139520 10628 tablet_service.cc:1514] Processing DeleteTablet for tablet 83d67392af814e43a0fa1bee970820b8 with delete_type TABLET_DATA_DELETED (replaced by tablet 39cdf419c8b24274bbad1fe4444dd73e) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:11.149673 11644 raft_consensus.cc:504] T 4e9f00abffe14046bf47a6cae7010690 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
W20260924 00:21:11.154698 11624 lock_manager.cc:586] Waited 2 seconds to obtain row lock on key '<redacted>' (key hash 12737376921701372339) tablet 00000000000000000000000000000000 cur holder 0x7b5800060600
I20260924 00:21:11.155078 10368 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_aa8bce70b4eb4f9a957369558580a1a2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:11.156474 10368 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aa8bce70b4eb4f9a957369558580a1a2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:11.162330 11039 tablet_bootstrap.cc:654] T cb51d5c6460049108cd7e1974821e578 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:11.134820 10798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e9f00abffe14046bf47a6cae7010690" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:11.163465 10651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e9f00abffe14046bf47a6cae7010690" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:21:11.152045 10794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e9f00abffe14046bf47a6cae7010690" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:21:11.163985 10651 raft_consensus.cc:3037] T 4e9f00abffe14046bf47a6cae7010690 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
W20260924 00:21:11.164283 10878 leader_election.cc:341] T 4e9f00abffe14046bf47a6cae7010690 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
W20260924 00:21:11.165163 10878 leader_election.cc:341] T 4e9f00abffe14046bf47a6cae7010690 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:11.166169 11040 ts_tablet_manager.cc:1428] T 0c583d9ba1414f8ea2e605b4b6b076f8 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.058s user 0.003s sys 0.002s
I20260924 00:21:11.152700 10778 tablet_service.cc:1514] Processing DeleteTablet for tablet 83d67392af814e43a0fa1bee970820b8 with delete_type TABLET_DATA_DELETED (replaced by tablet 39cdf419c8b24274bbad1fe4444dd73e) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:11.170123 10651 raft_consensus.cc:2454] T 4e9f00abffe14046bf47a6cae7010690 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 1.
I20260924 00:21:11.171271 10876 leader_election.cc:302] T 4e9f00abffe14046bf47a6cae7010690 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08
I20260924 00:21:11.171988 11668 raft_consensus.cc:2781] T 4e9f00abffe14046bf47a6cae7010690 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:11.172415 11668 raft_consensus.cc:686] T 4e9f00abffe14046bf47a6cae7010690 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Becoming Leader. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Running, Role: LEADER
I20260924 00:21:11.173650 10625 tablet_service.cc:1467] Processing CreateTablet for tablet 83c1ce21578341ad8ee28ec44c518dc3 (DEFAULT_TABLE table=default.Table_a214c5e4b9584a1fba04c74e906fe4d3 [id=28864c6aa1f143bfb7b49d98bff16f8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.173511 11668 consensus_queue.cc:237] T 4e9f00abffe14046bf47a6cae7010690 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:11.175050 11039 tablet_bootstrap.cc:492] T cb51d5c6460049108cd7e1974821e578 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:11.175416 11039 ts_tablet_manager.cc:1397] T cb51d5c6460049108cd7e1974821e578 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.057s user 0.009s sys 0.003s
I20260924 00:21:11.176457 10628 tablet_service.cc:1467] Processing CreateTablet for tablet ba81ed510bca46fda6935905fa0c29fb (DEFAULT_TABLE table=default.Table_5d4b70d8740f435bb43cb05808ec557e [id=dccf8774891e46649570c04d4ad36e6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.177978 10628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba81ed510bca46fda6935905fa0c29fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.178561 11644 leader_election.cc:288] T 4e9f00abffe14046bf47a6cae7010690 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:11.189841 10625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83c1ce21578341ad8ee28ec44c518dc3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:21:11.197772 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 83d67392af814e43a0fa1bee970820b8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 83d67392af814e43a0fa1bee970820b8 already in progress: creating tablet
I20260924 00:21:11.200589 10367 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 90d5d35a4a8d4bac99ec458d743a0478
I20260924 00:21:11.204285 11039 raft_consensus.cc:348] T cb51d5c6460049108cd7e1974821e578 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:11.205211 10780 tablet_service.cc:1514] Processing DeleteTablet for tablet 90d5d35a4a8d4bac99ec458d743a0478 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:10 UTC) from {username='slave'} at 127.0.0.1:35996
W20260924 00:21:11.206444 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 90d5d35a4a8d4bac99ec458d743a0478 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 90d5d35a4a8d4bac99ec458d743a0478 already in progress: creating tablet
I20260924 00:21:11.204897 11039 raft_consensus.cc:374] T cb51d5c6460049108cd7e1974821e578 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:11.207716 11039 raft_consensus.cc:729] T cb51d5c6460049108cd7e1974821e578 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:11.212272 11039 consensus_queue.cc:260] T cb51d5c6460049108cd7e1974821e578 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:11.216549 10908 tablet_service.cc:1514] Processing DeleteTablet for tablet 90d5d35a4a8d4bac99ec458d743a0478 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:10 UTC) from {username='slave'} at 127.0.0.1:35358
W20260924 00:21:11.218124 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 90d5d35a4a8d4bac99ec458d743a0478 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 90d5d35a4a8d4bac99ec458d743a0478 already in progress: creating tablet
I20260924 00:21:11.220743 11668 raft_consensus.cc:482] T cb51d5c6460049108cd7e1974821e578 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:11.221654 11668 raft_consensus.cc:504] T cb51d5c6460049108cd7e1974821e578 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:11.222815 10367 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_f9897a81fc084a25ae405522cf5c7ef9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:11.223445 11668 leader_election.cc:288] T cb51d5c6460049108cd7e1974821e578 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
W20260924 00:21:11.224699 10367 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f9897a81fc084a25ae405522cf5c7ef9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:11.224921 10794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb51d5c6460049108cd7e1974821e578" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:11.226429 10878 leader_election.cc:341] T cb51d5c6460049108cd7e1974821e578 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:11.229773 11039 ts_tablet_manager.cc:1428] T cb51d5c6460049108cd7e1974821e578 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.054s user 0.006s sys 0.000s
I20260924 00:21:11.230180 11040 tablet_bootstrap.cc:492] T a0f71049f6b3478da3b07ebc9293ce64 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:11.234548 10908 tablet_service.cc:1467] Processing CreateTablet for tablet e489ff048ced45d2b16112934d0bb3c1 (DEFAULT_TABLE table=default.Table_b07b221a38e14a93aa27e7c2dfe55549 [id=79232ff751084c1292ba92127d124c43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.235572 10908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e489ff048ced45d2b16112934d0bb3c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.240934 10780 tablet_service.cc:1467] Processing CreateTablet for tablet e489ff048ced45d2b16112934d0bb3c1 (DEFAULT_TABLE table=default.Table_b07b221a38e14a93aa27e7c2dfe55549 [id=79232ff751084c1292ba92127d124c43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.241904 10780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e489ff048ced45d2b16112934d0bb3c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.245674 10634 tablet_service.cc:1467] Processing CreateTablet for tablet e489ff048ced45d2b16112934d0bb3c1 (DEFAULT_TABLE table=default.Table_b07b221a38e14a93aa27e7c2dfe55549 [id=79232ff751084c1292ba92127d124c43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.246601 10634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e489ff048ced45d2b16112934d0bb3c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.250948 10909 tablet_service.cc:1467] Processing CreateTablet for tablet f72cedef5fe94074a8233e657e9346c8 (DEFAULT_TABLE table=default.Table_f0b5a94e5ae7437383840e76682bd5fa [id=ff0acca1a5c8440ca9e94f253f63df53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.251885 10909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f72cedef5fe94074a8233e657e9346c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.252156 10778 tablet_service.cc:1467] Processing CreateTablet for tablet f72cedef5fe94074a8233e657e9346c8 (DEFAULT_TABLE table=default.Table_f0b5a94e5ae7437383840e76682bd5fa [id=ff0acca1a5c8440ca9e94f253f63df53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.253124 10778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f72cedef5fe94074a8233e657e9346c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.262599 10627 tablet_service.cc:1467] Processing CreateTablet for tablet f72cedef5fe94074a8233e657e9346c8 (DEFAULT_TABLE table=default.Table_f0b5a94e5ae7437383840e76682bd5fa [id=ff0acca1a5c8440ca9e94f253f63df53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.263656 10627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f72cedef5fe94074a8233e657e9346c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.269523 11039 tablet_bootstrap.cc:492] T 3b8d74761b0b4fd0a22021d3870ec05c P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:11.211377 11049 ts_tablet_manager.cc:1918] T cb51d5c6460049108cd7e1974821e578 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:11.211804 10631 tablet_service.cc:1514] Processing DeleteTablet for tablet 90d5d35a4a8d4bac99ec458d743a0478 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:10 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:11.226897 10651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb51d5c6460049108cd7e1974821e578" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:11.277380 11049 log.cc:1196] T cb51d5c6460049108cd7e1974821e578 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/cb51d5c6460049108cd7e1974821e578
I20260924 00:21:11.278010 11049 ts_tablet_manager.cc:1939] T cb51d5c6460049108cd7e1974821e578 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
W20260924 00:21:11.278018 10876 leader_election.cc:341] T cb51d5c6460049108cd7e1974821e578 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Illegal state: Tablet not RUNNING: STOPPED
I20260924 00:21:11.278388 10876 leader_election.cc:302] T cb51d5c6460049108cd7e1974821e578 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670
I20260924 00:21:11.279457 11668 raft_consensus.cc:2726] T cb51d5c6460049108cd7e1974821e578 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:11.228101 11041 tablet_bootstrap.cc:492] T 675b9e33edcf4d4ea7fa428b476e3945 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:11.280241 11041 ts_tablet_manager.cc:1397] T 675b9e33edcf4d4ea7fa428b476e3945 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.290s user 0.007s sys 0.011s
I20260924 00:21:11.281955 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet cb51d5c6460049108cd7e1974821e578 (table default.Table_9172ff8a00a54ed581e96de0f0b44c3c [id=ed975cbe1bb04e3194882bed0f4ec0b1]) successfully deleted
I20260924 00:21:11.282269 11041 raft_consensus.cc:348] T 675b9e33edcf4d4ea7fa428b476e3945 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
W20260924 00:21:11.282603 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 0cff84bc75aa41aaac797ac183c7dd4a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0cff84bc75aa41aaac797ac183c7dd4a already in progress: creating tablet
I20260924 00:21:11.282855 11041 raft_consensus.cc:374] T 675b9e33edcf4d4ea7fa428b476e3945 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:11.283082 11041 raft_consensus.cc:729] T 675b9e33edcf4d4ea7fa428b476e3945 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
W20260924 00:21:11.283219 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 83d67392af814e43a0fa1bee970820b8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 83d67392af814e43a0fa1bee970820b8 already in progress: creating tablet
I20260924 00:21:11.283953 11041 consensus_queue.cc:260] T 675b9e33edcf4d4ea7fa428b476e3945 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:11.291851 10374 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet a6e4030e824047ca9cc92d98707d3335
I20260924 00:21:11.293881 10774 tablet_service.cc:1514] Processing DeleteTablet for tablet a6e4030e824047ca9cc92d98707d3335 with delete_type TABLET_DATA_DELETED (replaced by tablet 36b1bc216f964b898503712eda33eeb6) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:11.294183 10920 tablet_service.cc:1514] Processing DeleteTablet for tablet a6e4030e824047ca9cc92d98707d3335 with delete_type TABLET_DATA_DELETED (replaced by tablet 36b1bc216f964b898503712eda33eeb6) from {username='slave'} at 127.0.0.1:35358
W20260924 00:21:11.295315 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet a6e4030e824047ca9cc92d98707d3335 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a6e4030e824047ca9cc92d98707d3335 already in progress: creating tablet
W20260924 00:21:11.299613 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet a6e4030e824047ca9cc92d98707d3335 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a6e4030e824047ca9cc92d98707d3335 already in progress: creating tablet
I20260924 00:21:11.300558 11039 tablet_bootstrap.cc:654] T 3b8d74761b0b4fd0a22021d3870ec05c P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:11.301700 10374 catalog_manager.cc:6565] ReplaceTablet: tablet a6e4030e824047ca9cc92d98707d3335 deleted and replaced by tablet 36b1bc216f964b898503712eda33eeb6
I20260924 00:21:11.302073 11041 ts_tablet_manager.cc:1428] T 675b9e33edcf4d4ea7fa428b476e3945 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.022s user 0.004s sys 0.002s
I20260924 00:21:11.305130 10631 tablet_service.cc:1514] Processing DeleteTablet for tablet a6e4030e824047ca9cc92d98707d3335 with delete_type TABLET_DATA_DELETED (replaced by tablet 36b1bc216f964b898503712eda33eeb6) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:11.313503 11041 tablet_bootstrap.cc:492] T 5b43ce1fed0749b8b076b9b026914e07 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
W20260924 00:21:11.314179 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 90d5d35a4a8d4bac99ec458d743a0478 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 90d5d35a4a8d4bac99ec458d743a0478 already in progress: creating tablet
W20260924 00:21:11.318938 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet a6e4030e824047ca9cc92d98707d3335 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a6e4030e824047ca9cc92d98707d3335 already in progress: creating tablet
I20260924 00:21:11.321017 10374 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_d3365a0130ed438d9a05029476376802" } modify_external_catalogs: true
I20260924 00:21:11.321537 10374 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_d3365a0130ed438d9a05029476376802" } modify_external_catalogs: true
I20260924 00:21:11.334113 11041 tablet_bootstrap.cc:654] T 5b43ce1fed0749b8b076b9b026914e07 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:11.339390 11039 tablet_bootstrap.cc:492] T 3b8d74761b0b4fd0a22021d3870ec05c P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:11.339831 11039 ts_tablet_manager.cc:1397] T 3b8d74761b0b4fd0a22021d3870ec05c P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.070s user 0.005s sys 0.004s
I20260924 00:21:11.341962 11039 raft_consensus.cc:348] T 3b8d74761b0b4fd0a22021d3870ec05c P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:11.342576 11039 raft_consensus.cc:374] T 3b8d74761b0b4fd0a22021d3870ec05c P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:11.342799 11039 raft_consensus.cc:729] T 3b8d74761b0b4fd0a22021d3870ec05c P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:11.343569 11039 consensus_queue.cc:260] T 3b8d74761b0b4fd0a22021d3870ec05c P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:11.345822 11039 ts_tablet_manager.cc:1428] T 3b8d74761b0b4fd0a22021d3870ec05c P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260924 00:21:11.353590 11039 tablet_bootstrap.cc:492] T 10aa64ed664f40629f5f134052df8f5f P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:11.363348 10909 tablet_service.cc:1467] Processing CreateTablet for tablet ea7158417c944cbe9f46121964a87789 (DEFAULT_TABLE table=default.Table_dbe5555ea7824aabb73edf7e94b89b03 [id=cceeb868b61c47e7a39d907e4f88000f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.363886 11041 tablet_bootstrap.cc:492] T 5b43ce1fed0749b8b076b9b026914e07 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:11.364240 11041 ts_tablet_manager.cc:1397] T 5b43ce1fed0749b8b076b9b026914e07 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.051s user 0.008s sys 0.000s
I20260924 00:21:11.364302 10909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea7158417c944cbe9f46121964a87789. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.365113 11040 tablet_bootstrap.cc:654] T a0f71049f6b3478da3b07ebc9293ce64 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
W20260924 00:21:11.365728 10365 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0beb7ec3095549219734befe57d9daec consensus state has no leader
I20260924 00:21:11.366255 11041 raft_consensus.cc:348] T 5b43ce1fed0749b8b076b9b026914e07 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:11.366859 11041 raft_consensus.cc:374] T 5b43ce1fed0749b8b076b9b026914e07 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:11.367081 11041 raft_consensus.cc:729] T 5b43ce1fed0749b8b076b9b026914e07 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:11.369065 11039 tablet_bootstrap.cc:654] T 10aa64ed664f40629f5f134052df8f5f P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:11.375204 11041 consensus_queue.cc:260] T 5b43ce1fed0749b8b076b9b026914e07 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:11.378042 10778 tablet_service.cc:1467] Processing CreateTablet for tablet ea7158417c944cbe9f46121964a87789 (DEFAULT_TABLE table=default.Table_dbe5555ea7824aabb73edf7e94b89b03 [id=cceeb868b61c47e7a39d907e4f88000f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.378964 10778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea7158417c944cbe9f46121964a87789. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.380386 10780 tablet_service.cc:1467] Processing CreateTablet for tablet 6f2195c79a9342bcbd1de472b56df83c (DEFAULT_TABLE table=default.Table_9b48f9449d9149128f9d014b582394ac [id=f2dc666553b843528f09119e75874b1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.381254 10780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f2195c79a9342bcbd1de472b56df83c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.381966 10774 tablet_service.cc:1467] Processing CreateTablet for tablet 61ca7994dac34681847fe5748509fe24 (DEFAULT_TABLE table=default.Table_a9fb0b9afed748b38640e91ac8eeedd2 [id=ee34e2e0b3564d109c4737f6093db745]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.381914 10920 tablet_service.cc:1467] Processing CreateTablet for tablet 6f2195c79a9342bcbd1de472b56df83c (DEFAULT_TABLE table=default.Table_9b48f9449d9149128f9d014b582394ac [id=f2dc666553b843528f09119e75874b1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.382777 10920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f2195c79a9342bcbd1de472b56df83c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.382776 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61ca7994dac34681847fe5748509fe24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.386052 11041 ts_tablet_manager.cc:1428] T 5b43ce1fed0749b8b076b9b026914e07 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.022s user 0.006s sys 0.000s
I20260924 00:21:11.385892 10634 tablet_service.cc:1467] Processing CreateTablet for tablet 6f2195c79a9342bcbd1de472b56df83c (DEFAULT_TABLE table=default.Table_9b48f9449d9149128f9d014b582394ac [id=f2dc666553b843528f09119e75874b1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.390712 11041 tablet_bootstrap.cc:492] T a62b85ebb2b84ae19cad3e3e2dd07a59 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:11.395239 11041 tablet_bootstrap.cc:654] T a62b85ebb2b84ae19cad3e3e2dd07a59 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:11.404726 10365 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_c6e518c165b3408d80c30626859c2086" } modify_external_catalogs: true
I20260924 00:21:11.405264 10365 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_c6e518c165b3408d80c30626859c2086" } modify_external_catalogs: true
W20260924 00:21:11.418022 10366 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5203b62bde2546e5b99034afde9c0813 consensus state has no leader
I20260924 00:21:11.425556 11039 tablet_bootstrap.cc:492] T 10aa64ed664f40629f5f134052df8f5f P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:11.425920 11039 ts_tablet_manager.cc:1397] T 10aa64ed664f40629f5f134052df8f5f P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.072s user 0.005s sys 0.003s
I20260924 00:21:11.428052 11039 raft_consensus.cc:348] T 10aa64ed664f40629f5f134052df8f5f P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:11.431317 10628 tablet_service.cc:1467] Processing CreateTablet for tablet ea7158417c944cbe9f46121964a87789 (DEFAULT_TABLE table=default.Table_dbe5555ea7824aabb73edf7e94b89b03 [id=cceeb868b61c47e7a39d907e4f88000f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.432503 10634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f2195c79a9342bcbd1de472b56df83c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.428632 11039 raft_consensus.cc:374] T 10aa64ed664f40629f5f134052df8f5f P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:11.433104 11039 raft_consensus.cc:729] T 10aa64ed664f40629f5f134052df8f5f P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:11.433698 11039 consensus_queue.cc:260] T 10aa64ed664f40629f5f134052df8f5f P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:11.449048 10909 tablet_service.cc:1467] Processing CreateTablet for tablet 61ca7994dac34681847fe5748509fe24 (DEFAULT_TABLE table=default.Table_a9fb0b9afed748b38640e91ac8eeedd2 [id=ee34e2e0b3564d109c4737f6093db745]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.454468 10780 tablet_service.cc:1467] Processing CreateTablet for tablet 0beb7ec3095549219734befe57d9daec (DEFAULT_TABLE table=default.Table_ceb867d167eb4df8bb2653ede8e37644 [id=b8ac1418c7d644b7b635d6cb28c100b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.455464 10780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0beb7ec3095549219734befe57d9daec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.455835 10778 tablet_service.cc:1467] Processing CreateTablet for tablet 40bfe924a639439d956a387fb6c5e324 (DEFAULT_TABLE table=default.Table_9eeed352b33c49649d52ad128cb67203 [id=0b05d4353e924845834050bc136e812a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.461606 10778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40bfe924a639439d956a387fb6c5e324. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.451032 11039 ts_tablet_manager.cc:1428] T 10aa64ed664f40629f5f134052df8f5f P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.025s user 0.004s sys 0.001s
I20260924 00:21:11.462670 10631 tablet_service.cc:1467] Processing CreateTablet for tablet 61ca7994dac34681847fe5748509fe24 (DEFAULT_TABLE table=default.Table_a9fb0b9afed748b38640e91ac8eeedd2 [id=ee34e2e0b3564d109c4737f6093db745]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.464296 10632 tablet_service.cc:1467] Processing CreateTablet for tablet 40bfe924a639439d956a387fb6c5e324 (DEFAULT_TABLE table=default.Table_9eeed352b33c49649d52ad128cb67203 [id=0b05d4353e924845834050bc136e812a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.465067 10907 tablet_service.cc:1467] Processing CreateTablet for tablet 0beb7ec3095549219734befe57d9daec (DEFAULT_TABLE table=default.Table_ceb867d167eb4df8bb2653ede8e37644 [id=b8ac1418c7d644b7b635d6cb28c100b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.465754 11040 tablet_bootstrap.cc:492] T a0f71049f6b3478da3b07ebc9293ce64 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:11.466142 11040 ts_tablet_manager.cc:1397] T a0f71049f6b3478da3b07ebc9293ce64 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.236s user 0.004s sys 0.004s
I20260924 00:21:11.465065 10909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61ca7994dac34681847fe5748509fe24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.468231 11040 raft_consensus.cc:348] T a0f71049f6b3478da3b07ebc9293ce64 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:11.467898 11039 tablet_bootstrap.cc:492] T e937cee22326474598cef5c1d09b59c2 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:11.471350 10774 tablet_service.cc:1467] Processing CreateTablet for tablet f3abf4ab0f36450c9effcca01aff0719 (DEFAULT_TABLE table=default.Table_4541883eff524699a7226b51ad3b9727 [id=896bcf73282349db8fab4222fc76ca71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.472170 10988 heartbeater.cc:347] Connected to a master server at 127.10.4.124:35345
I20260924 00:21:11.472425 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3abf4ab0f36450c9effcca01aff0719. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.473227 10907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0beb7ec3095549219734befe57d9daec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.483515 10631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61ca7994dac34681847fe5748509fe24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.485111 11041 tablet_bootstrap.cc:492] T a62b85ebb2b84ae19cad3e3e2dd07a59 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:11.485473 11041 ts_tablet_manager.cc:1397] T a62b85ebb2b84ae19cad3e3e2dd07a59 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.095s user 0.006s sys 0.002s
I20260924 00:21:11.487884 11041 raft_consensus.cc:348] T a62b85ebb2b84ae19cad3e3e2dd07a59 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:11.488472 11041 raft_consensus.cc:374] T a62b85ebb2b84ae19cad3e3e2dd07a59 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:11.488680 11041 raft_consensus.cc:729] T a62b85ebb2b84ae19cad3e3e2dd07a59 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:11.489584 10632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40bfe924a639439d956a387fb6c5e324. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:21:11.507282 11697 log.cc:924] Time spent T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92: Append to log took a long time: real 0.192s user 0.021s sys 0.004s
I20260924 00:21:11.510636 11039 tablet_bootstrap.cc:654] T e937cee22326474598cef5c1d09b59c2 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:11.513087 10628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea7158417c944cbe9f46121964a87789. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.468820 11040 raft_consensus.cc:374] T a0f71049f6b3478da3b07ebc9293ce64 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:11.518203 10626 tablet_service.cc:1467] Processing CreateTablet for tablet 0beb7ec3095549219734befe57d9daec (DEFAULT_TABLE table=default.Table_ceb867d167eb4df8bb2653ede8e37644 [id=b8ac1418c7d644b7b635d6cb28c100b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.518465 11040 raft_consensus.cc:729] T a0f71049f6b3478da3b07ebc9293ce64 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:11.509205 11041 consensus_queue.cc:260] T a62b85ebb2b84ae19cad3e3e2dd07a59 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:11.519280 10626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0beb7ec3095549219734befe57d9daec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.519136 11040 consensus_queue.cc:260] T a0f71049f6b3478da3b07ebc9293ce64 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:11.521183 11040 ts_tablet_manager.cc:1428] T a0f71049f6b3478da3b07ebc9293ce64 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.055s user 0.003s sys 0.003s
I20260924 00:21:11.533586 11040 tablet_bootstrap.cc:492] T 72d3bb80e1034881b94eb75de5d900c2 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:11.536989 11668 raft_consensus.cc:482] T b36cbb6520de4df58b88cfa53c20ee7a P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:11.537431 11668 raft_consensus.cc:504] T b36cbb6520de4df58b88cfa53c20ee7a P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:11.539242 11668 leader_election.cc:288] T b36cbb6520de4df58b88cfa53c20ee7a P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:11.539618 10915 tablet_service.cc:1467] Processing CreateTablet for tablet f3abf4ab0f36450c9effcca01aff0719 (DEFAULT_TABLE table=default.Table_4541883eff524699a7226b51ad3b9727 [id=896bcf73282349db8fab4222fc76ca71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.541015 10775 tablet_service.cc:1467] Processing CreateTablet for tablet ba31383ba3a04edab4c250c4b3146185 (DEFAULT_TABLE table=default.Table_e3fed8d230874b52bf545da40daf0c48 [id=2613a8863ddb477498b493dfcc76114a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.541003 10630 tablet_service.cc:1467] Processing CreateTablet for tablet f3abf4ab0f36450c9effcca01aff0719 (DEFAULT_TABLE table=default.Table_4541883eff524699a7226b51ad3b9727 [id=896bcf73282349db8fab4222fc76ca71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.541919 10775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba31383ba3a04edab4c250c4b3146185. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.542045 10630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3abf4ab0f36450c9effcca01aff0719. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.545296 10794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b36cbb6520de4df58b88cfa53c20ee7a" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:11.546543 10876 leader_election.cc:341] T b36cbb6520de4df58b88cfa53c20ee7a P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: b36cbb6520de4df58b88cfa53c20ee7a
W20260924 00:21:11.546911 10878 leader_election.cc:341] T b36cbb6520de4df58b88cfa53c20ee7a P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:11.547240 10878 leader_election.cc:302] T b36cbb6520de4df58b88cfa53c20ee7a P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670
I20260924 00:21:11.548070 11668 raft_consensus.cc:2726] T b36cbb6520de4df58b88cfa53c20ee7a P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:11.548089 10912 tablet_service.cc:1467] Processing CreateTablet for tablet ba31383ba3a04edab4c250c4b3146185 (DEFAULT_TABLE table=default.Table_e3fed8d230874b52bf545da40daf0c48 [id=2613a8863ddb477498b493dfcc76114a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.555447 10625 tablet_service.cc:1467] Processing CreateTablet for tablet ba31383ba3a04edab4c250c4b3146185 (DEFAULT_TABLE table=default.Table_e3fed8d230874b52bf545da40daf0c48 [id=2613a8863ddb477498b493dfcc76114a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.556468 10625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba31383ba3a04edab4c250c4b3146185. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.569341 11668 raft_consensus.cc:482] T 59a0a950a8144e99b2ed47be9a567d9a P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:11.569769 11668 raft_consensus.cc:504] T 59a0a950a8144e99b2ed47be9a567d9a P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:11.572043 11668 leader_election.cc:288] T 59a0a950a8144e99b2ed47be9a567d9a P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:11.572789 10794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59a0a950a8144e99b2ed47be9a567d9a" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:11.574030 10878 leader_election.cc:341] T 59a0a950a8144e99b2ed47be9a567d9a P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
W20260924 00:21:11.574540 10876 leader_election.cc:341] T 59a0a950a8144e99b2ed47be9a567d9a P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: 59a0a950a8144e99b2ed47be9a567d9a
I20260924 00:21:11.574875 10876 leader_election.cc:302] T 59a0a950a8144e99b2ed47be9a567d9a P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670
I20260924 00:21:11.575744 11668 raft_consensus.cc:2726] T 59a0a950a8144e99b2ed47be9a567d9a P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:11.579080 10778 tablet_service.cc:1467] Processing CreateTablet for tablet 13310c1530604e5595bf14dca31242c1 (DEFAULT_TABLE table=default.Table_81b3a0208acd453c8a35957f47f25ff3 [id=14d056bc2e7849ebb6e6ca7966e11324]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.580216 11637 raft_consensus.cc:482] T 0c583d9ba1414f8ea2e605b4b6b076f8 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:11.580615 11637 raft_consensus.cc:504] T 0c583d9ba1414f8ea2e605b4b6b076f8 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:11.580850 10906 tablet_service.cc:1467] Processing CreateTablet for tablet 13310c1530604e5595bf14dca31242c1 (DEFAULT_TABLE table=default.Table_81b3a0208acd453c8a35957f47f25ff3 [id=14d056bc2e7849ebb6e6ca7966e11324]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.581694 10920 tablet_service.cc:1467] Processing CreateTablet for tablet 40bfe924a639439d956a387fb6c5e324 (DEFAULT_TABLE table=default.Table_9eeed352b33c49649d52ad128cb67203 [id=0b05d4353e924845834050bc136e812a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.581897 10780 tablet_service.cc:1467] Processing CreateTablet for tablet fd0c3f766a464116aa777b849cf4acc2 (DEFAULT_TABLE table=default.Table_3f26bb58c6374373a24eb69faf5d68c6 [id=c5d285065f184765b488817af0b5d87d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.582345 11637 leader_election.cc:288] T 0c583d9ba1414f8ea2e605b4b6b076f8 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:11.584172 10910 tablet_service.cc:1467] Processing CreateTablet for tablet fd0c3f766a464116aa777b849cf4acc2 (DEFAULT_TABLE table=default.Table_3f26bb58c6374373a24eb69faf5d68c6 [id=c5d285065f184765b488817af0b5d87d]), partition=RANGE (key) PARTITION UNBOUNDED
W20260924 00:21:11.584404 10732 leader_election.cc:341] T 0c583d9ba1414f8ea2e605b4b6b076f8 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Not found: Tablet not found: 0c583d9ba1414f8ea2e605b4b6b076f8
I20260924 00:21:11.586967 10627 tablet_service.cc:1467] Processing CreateTablet for tablet fd0c3f766a464116aa777b849cf4acc2 (DEFAULT_TABLE table=default.Table_3f26bb58c6374373a24eb69faf5d68c6 [id=c5d285065f184765b488817af0b5d87d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.588017 10627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd0c3f766a464116aa777b849cf4acc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.588549 11040 tablet_bootstrap.cc:654] T 72d3bb80e1034881b94eb75de5d900c2 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
W20260924 00:21:11.592702 10733 leader_election.cc:341] T 0c583d9ba1414f8ea2e605b4b6b076f8 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: 0c583d9ba1414f8ea2e605b4b6b076f8
I20260924 00:21:11.593108 10733 leader_election.cc:302] T 0c583d9ba1414f8ea2e605b4b6b076f8 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08; no voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:11.593997 11637 raft_consensus.cc:2726] T 0c583d9ba1414f8ea2e605b4b6b076f8 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:11.596321 10366 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_49cf1873c1f4414fa918497b3a48015b" } new_table_name: "default.Table_d1104bf9437443d48b9e769fcf06d5ec" modify_external_catalogs: true
I20260924 00:21:11.603067 11041 ts_tablet_manager.cc:1428] T a62b85ebb2b84ae19cad3e3e2dd07a59 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.117s user 0.006s sys 0.000s
I20260924 00:21:11.604223 11041 tablet_bootstrap.cc:492] T 3c850d0474414d2486cd7bbd6568e936 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:11.607177 10915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3abf4ab0f36450c9effcca01aff0719. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.613430 10912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba31383ba3a04edab4c250c4b3146185. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.621876 10906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13310c1530604e5595bf14dca31242c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.631098 10778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13310c1530604e5595bf14dca31242c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.631736 10780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd0c3f766a464116aa777b849cf4acc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.645040 11040 tablet_bootstrap.cc:492] T 72d3bb80e1034881b94eb75de5d900c2 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:11.650373 10369 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_6cf69012192f4f1087e29db70ae8c282" } new_table_name: "default.Table_b36ac15778564d608790f586457c6373" modify_external_catalogs: true
I20260924 00:21:11.645433 11040 ts_tablet_manager.cc:1397] T 72d3bb80e1034881b94eb75de5d900c2 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.112s user 0.009s sys 0.003s
I20260924 00:21:11.665669 11040 raft_consensus.cc:348] T 72d3bb80e1034881b94eb75de5d900c2 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:11.666296 11040 raft_consensus.cc:374] T 72d3bb80e1034881b94eb75de5d900c2 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:11.666522 11040 raft_consensus.cc:729] T 72d3bb80e1034881b94eb75de5d900c2 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:11.670539 11040 consensus_queue.cc:260] T 72d3bb80e1034881b94eb75de5d900c2 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:11.672403 10909 tablet_service.cc:1467] Processing CreateTablet for tablet 41bdfd499c6b42a2a7516c0157efddbb (DEFAULT_TABLE table=default.Table_b5c9933d8e0c481290863b60a642e8e7 [id=036fc1e2f392468ca0e9ce72610f3620]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.673416 10909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41bdfd499c6b42a2a7516c0157efddbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.675860 10778 tablet_service.cc:1467] Processing CreateTablet for tablet 41bdfd499c6b42a2a7516c0157efddbb (DEFAULT_TABLE table=default.Table_b5c9933d8e0c481290863b60a642e8e7 [id=036fc1e2f392468ca0e9ce72610f3620]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.687505 11641 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b5aa3debe1cd45e19a6bbd0070913ac3)
I20260924 00:21:11.688151 11641 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } }
I20260924 00:21:11.689505 10910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd0c3f766a464116aa777b849cf4acc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.689332 11611 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5aa3debe1cd45e19a6bbd0070913ac3. Latest consensus state: current_term: 2 leader_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } } }
I20260924 00:21:11.690049 11611 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:21:11.697312 10920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40bfe924a639439d956a387fb6c5e324. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.701974 11710 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260924 00:21:11.702924 11039 tablet_bootstrap.cc:492] T e937cee22326474598cef5c1d09b59c2 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:11.703296 11039 ts_tablet_manager.cc:1397] T e937cee22326474598cef5c1d09b59c2 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.236s user 0.005s sys 0.010s
I20260924 00:21:11.706457 11039 raft_consensus.cc:348] T e937cee22326474598cef5c1d09b59c2 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:11.706875 10908 tablet_service.cc:1467] Processing CreateTablet for tablet 1acc16e2d3e14ab092da083b720cc29a (DEFAULT_TABLE table=default.Table_60e366f2e65b432da448892cea15e440 [id=8ae806d7df6c4158959efbe958fd4bcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.708765 10908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1acc16e2d3e14ab092da083b720cc29a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.707046 11039 raft_consensus.cc:374] T e937cee22326474598cef5c1d09b59c2 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:11.711979 11039 raft_consensus.cc:729] T e937cee22326474598cef5c1d09b59c2 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:11.712677 11039 consensus_queue.cc:260] T e937cee22326474598cef5c1d09b59c2 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:11.717458 11039 ts_tablet_manager.cc:1428] T e937cee22326474598cef5c1d09b59c2 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.014s user 0.006s sys 0.000s
I20260924 00:21:11.718338 11039 tablet_bootstrap.cc:492] T 675b9e33edcf4d4ea7fa428b476e3945 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:11.683612 11040 ts_tablet_manager.cc:1428] T 72d3bb80e1034881b94eb75de5d900c2 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.029s user 0.004s sys 0.001s
I20260924 00:21:11.691990 10845 heartbeater.cc:347] Connected to a master server at 127.10.4.124:35345
I20260924 00:21:11.719455 10372 catalog_manager.cc:6040] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: default.Table_b7a5eb7bd9fe4e0483e2d47d6182e46d [id=f79ca8d11aa04d5f8f72d58b798ed4ea] alter complete (version 2)
I20260924 00:21:11.719939 11040 tablet_bootstrap.cc:492] T 4e9f00abffe14046bf47a6cae7010690 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:11.709460 11641 leader_election.cc:288] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1d68651913494ce8afee3c136b2da9cc (127.10.4.126:34479), b5aa3debe1cd45e19a6bbd0070913ac3 (127.10.4.125:34487)
I20260924 00:21:11.705726 10775 tablet_service.cc:1467] Processing CreateTablet for tablet 1acc16e2d3e14ab092da083b720cc29a (DEFAULT_TABLE table=default.Table_60e366f2e65b432da448892cea15e440 [id=8ae806d7df6c4158959efbe958fd4bcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.728706 10626 tablet_service.cc:1467] Processing CreateTablet for tablet 13310c1530604e5595bf14dca31242c1 (DEFAULT_TABLE table=default.Table_81b3a0208acd453c8a35957f47f25ff3 [id=14d056bc2e7849ebb6e6ca7966e11324]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.729358 10778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41bdfd499c6b42a2a7516c0157efddbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.729737 10626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13310c1530604e5595bf14dca31242c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.732916 10775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1acc16e2d3e14ab092da083b720cc29a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.749872 10915 tablet_service.cc:1467] Processing CreateTablet for tablet 22640bf1c2304d978e66c002d74aab51 (DEFAULT_TABLE table=default.Table_0a7f75ce29d74b7da36e39b072bffccc [id=332608cae68d4f51aff9ab21f3bb9e12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.751964 10780 tablet_service.cc:1467] Processing CreateTablet for tablet 4bbc6df13fe14ecbae236bed68f037dc (DEFAULT_TABLE table=default.Table_e110cb436d444a319359e45630f8e2bb [id=d5ce3c6422db4e42912aabb3fd6c0e63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.753186 10780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bbc6df13fe14ecbae236bed68f037dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.756496 11459 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "48fbd603abe146c585d15196e1b5be08" instance_seqno: 1790209253125740) as {username='slave'} at 127.10.4.66:33619; Asking this server to re-register.
I20260924 00:21:11.760887 10845 heartbeater.cc:464] Registering TS with master...
I20260924 00:21:11.762884 10774 tablet_service.cc:1467] Processing CreateTablet for tablet 22640bf1c2304d978e66c002d74aab51 (DEFAULT_TABLE table=default.Table_0a7f75ce29d74b7da36e39b072bffccc [id=332608cae68d4f51aff9ab21f3bb9e12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.765565 10625 tablet_service.cc:1467] Processing CreateTablet for tablet 1acc16e2d3e14ab092da083b720cc29a (DEFAULT_TABLE table=default.Table_60e366f2e65b432da448892cea15e440 [id=8ae806d7df6c4158959efbe958fd4bcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.766497 10625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1acc16e2d3e14ab092da083b720cc29a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.764909 10634 tablet_service.cc:1467] Processing CreateTablet for tablet 41bdfd499c6b42a2a7516c0157efddbb (DEFAULT_TABLE table=default.Table_b5c9933d8e0c481290863b60a642e8e7 [id=036fc1e2f392468ca0e9ce72610f3620]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.769694 10634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41bdfd499c6b42a2a7516c0157efddbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.779502 10367 master_service.cc:946] ReplaceTablet: received request to replace tablet 5203b62bde2546e5b99034afde9c0813 from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:11.784008 10915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22640bf1c2304d978e66c002d74aab51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.786053 11456 ts_manager.cc:194] Registered new tserver with Master: 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:21:11.783932 11040 tablet_bootstrap.cc:654] T 4e9f00abffe14046bf47a6cae7010690 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:11.793969 10374 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet d71ce6053fdd4d53b3e0e596148d4c5c
I20260924 00:21:11.797042 10779 tablet_service.cc:1514] Processing DeleteTablet for tablet d71ce6053fdd4d53b3e0e596148d4c5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:11 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:11.799150 10777 tablet_service.cc:1467] Processing CreateTablet for tablet 8e99dc66ce9e41ef98b6b24fa41c1b36 (DEFAULT_TABLE table=default.Table_ee73693b7a6e46a3af53f7828da00f9e [id=9359b05533064c38b9bd589c0bb06471]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.674597 11693 raft_consensus.cc:482] T 5b43ce1fed0749b8b076b9b026914e07 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:11.802338 10773 tablet_service.cc:1467] Processing CreateTablet for tablet 45ba52ea61364b988ce77b265f8c7a79 (DEFAULT_TABLE table=default.Table_a5a83435a7fb47ce8083022c0686b74a [id=a587a50b0c204843a41acacc45c2b150]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.760023 10906 tablet_service.cc:1467] Processing CreateTablet for tablet 4bbc6df13fe14ecbae236bed68f037dc (DEFAULT_TABLE table=default.Table_e110cb436d444a319359e45630f8e2bb [id=d5ce3c6422db4e42912aabb3fd6c0e63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.803619 10906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bbc6df13fe14ecbae236bed68f037dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.805181 11637 raft_consensus.cc:482] T a0f71049f6b3478da3b07ebc9293ce64 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:11.805557 11637 raft_consensus.cc:504] T a0f71049f6b3478da3b07ebc9293ce64 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:11.806604 10631 tablet_service.cc:1467] Processing CreateTablet for tablet 22640bf1c2304d978e66c002d74aab51 (DEFAULT_TABLE table=default.Table_0a7f75ce29d74b7da36e39b072bffccc [id=332608cae68d4f51aff9ab21f3bb9e12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.807327 11637 leader_election.cc:288] T a0f71049f6b3478da3b07ebc9293ce64 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:11.804020 10909 tablet_service.cc:1514] Processing DeleteTablet for tablet d71ce6053fdd4d53b3e0e596148d4c5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:11 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:11.807484 10631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22640bf1c2304d978e66c002d74aab51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.805565 10912 tablet_service.cc:1467] Processing CreateTablet for tablet 8e99dc66ce9e41ef98b6b24fa41c1b36 (DEFAULT_TABLE table=default.Table_ee73693b7a6e46a3af53f7828da00f9e [id=9359b05533064c38b9bd589c0bb06471]), partition=RANGE (key) PARTITION UNBOUNDED
W20260924 00:21:11.810758 10733 leader_election.cc:341] T a0f71049f6b3478da3b07ebc9293ce64 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: a0f71049f6b3478da3b07ebc9293ce64
I20260924 00:21:11.811173 10632 tablet_service.cc:1467] Processing CreateTablet for tablet 4bbc6df13fe14ecbae236bed68f037dc (DEFAULT_TABLE table=default.Table_e110cb436d444a319359e45630f8e2bb [id=d5ce3c6422db4e42912aabb3fd6c0e63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.811411 10912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e99dc66ce9e41ef98b6b24fa41c1b36. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:21:11.811652 10732 leader_election.cc:341] T a0f71049f6b3478da3b07ebc9293ce64 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Not found: Tablet not found: a0f71049f6b3478da3b07ebc9293ce64
I20260924 00:21:11.812044 10632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bbc6df13fe14ecbae236bed68f037dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.812000 10732 leader_election.cc:302] T a0f71049f6b3478da3b07ebc9293ce64 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08; no voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:11.812901 11637 raft_consensus.cc:2726] T a0f71049f6b3478da3b07ebc9293ce64 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:11.820863 10374 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_e207dff5fb014842a20e58edf524849e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:11.822290 10374 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e207dff5fb014842a20e58edf524849e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:11.831141 11644 raft_consensus.cc:482] T 3b8d74761b0b4fd0a22021d3870ec05c P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:11.831554 11644 raft_consensus.cc:504] T 3b8d74761b0b4fd0a22021d3870ec05c P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:11.838109 10629 tablet_service.cc:1467] Processing CreateTablet for tablet 8e99dc66ce9e41ef98b6b24fa41c1b36 (DEFAULT_TABLE table=default.Table_ee73693b7a6e46a3af53f7828da00f9e [id=9359b05533064c38b9bd589c0bb06471]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.839241 10629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e99dc66ce9e41ef98b6b24fa41c1b36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.841311 10635 tablet_service.cc:1514] Processing DeleteTablet for tablet d71ce6053fdd4d53b3e0e596148d4c5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:11 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:11.842156 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b8d74761b0b4fd0a22021d3870ec05c" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:11.842736 10654 raft_consensus.cc:2454] T 3b8d74761b0b4fd0a22021d3870ec05c P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
I20260924 00:21:11.845111 10876 leader_election.cc:302] T 3b8d74761b0b4fd0a22021d3870ec05c P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:11.846103 11668 raft_consensus.cc:2781] T 3b8d74761b0b4fd0a22021d3870ec05c P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:11.846457 11668 raft_consensus.cc:482] T 3b8d74761b0b4fd0a22021d3870ec05c P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:11.846730 11668 raft_consensus.cc:3037] T 3b8d74761b0b4fd0a22021d3870ec05c P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:11.848696 10368 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_e15931b8ec344adebc1ba60fa4be6f9b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:11.850208 10368 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e15931b8ec344adebc1ba60fa4be6f9b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:11.855960 11644 leader_election.cc:288] T 3b8d74761b0b4fd0a22021d3870ec05c P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:21:11.856139 11041 tablet_bootstrap.cc:654] T 3c850d0474414d2486cd7bbd6568e936 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:11.857146 10365 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 1a7eb76a787a447793fd04686f748d1e
W20260924 00:21:11.859656 10366 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 52c46283934a4c2180f15c7fc38b91a9 consensus state has no leader
I20260924 00:21:11.808337 11693 raft_consensus.cc:504] T 5b43ce1fed0749b8b076b9b026914e07 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
W20260924 00:21:11.862248 10369 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7673597f1afc43198e86929a2939269a consensus state has no leader
I20260924 00:21:11.863159 10779 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a7eb76a787a447793fd04686f748d1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:11 UTC) from {username='slave'} at 127.0.0.1:35996
W20260924 00:21:11.866050 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet d71ce6053fdd4d53b3e0e596148d4c5c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d71ce6053fdd4d53b3e0e596148d4c5c already in progress: creating tablet
I20260924 00:21:11.866495 10910 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a7eb76a787a447793fd04686f748d1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:11 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:11.868513 11040 tablet_bootstrap.cc:492] T 4e9f00abffe14046bf47a6cae7010690 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:11.868979 11040 ts_tablet_manager.cc:1397] T 4e9f00abffe14046bf47a6cae7010690 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.149s user 0.007s sys 0.005s
I20260924 00:21:11.871080 11040 raft_consensus.cc:348] T 4e9f00abffe14046bf47a6cae7010690 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
W20260924 00:21:11.877379 10358 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 52c46283934a4c2180f15c7fc38b91a9 (table default.Table_d1104bf9437443d48b9e769fcf06d5ec [id=5816a524b40d48629911e69babd7a80a], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 52c46283934a4c2180f15c7fc38b91a9 consensus state has no leader
I20260924 00:21:11.877983 11644 raft_consensus.cc:482] T 6bcfd729e9bd4040b3384ab24164e80d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:11.878588 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b43ce1fed0749b8b076b9b026914e07" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:11.878540 11644 raft_consensus.cc:504] T 6bcfd729e9bd4040b3384ab24164e80d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:11.880358 10633 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a7eb76a787a447793fd04686f748d1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:11 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:11.862972 11693 leader_election.cc:288] T 5b43ce1fed0749b8b076b9b026914e07 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:11.881677 10630 tablet_service.cc:1467] Processing CreateTablet for tablet 45ba52ea61364b988ce77b265f8c7a79 (DEFAULT_TABLE table=default.Table_a5a83435a7fb47ce8083022c0686b74a [id=a587a50b0c204843a41acacc45c2b150]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.880756 10308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" candidate_term: 3 candidate_status { last_received { term: 2 index: 322 } } ignore_live_leader: false dest_uuid: "1d68651913494ce8afee3c136b2da9cc" is_pre_election: true
I20260924 00:21:11.882411 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bcfd729e9bd4040b3384ab24164e80d" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:11.882676 10630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45ba52ea61364b988ce77b265f8c7a79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.882920 10654 raft_consensus.cc:2454] T 6bcfd729e9bd4040b3384ab24164e80d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
I20260924 00:21:11.883872 10876 leader_election.cc:302] T 6bcfd729e9bd4040b3384ab24164e80d P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:11.887748 11668 raft_consensus.cc:504] T 3b8d74761b0b4fd0a22021d3870ec05c P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:11.889307 10365 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_204d831d7f94465c8b58927b4988ef31"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:11.890682 10365 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_204d831d7f94465c8b58927b4988ef31 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:11.894722 11456 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "b10a9082a37b46f09d581944914c1670" instance_seqno: 1790209251321624) as {username='slave'} at 127.10.4.65:49739; Asking this server to re-register.
I20260924 00:21:11.902617 10701 heartbeater.cc:464] Registering TS with master...
I20260924 00:21:11.914232 10369 master_service.cc:946] ReplaceTablet: received request to replace tablet fcf651adb1d3459facfe3a8df2a91ed7 from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:11.916621 10633 tablet_service.cc:1467] Processing CreateTablet for tablet e97db32bea1a4cf1bea82d99c8db8147 (DEFAULT_TABLE table=default.Table_8823ba2975054860816e34a4fa104be8 [id=556a0c70dd484c5aaa120eb5f0dfe8e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.917596 10633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e97db32bea1a4cf1bea82d99c8db8147. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.926074 10636 tablet_service.cc:1467] Processing CreateTablet for tablet 5630f5ea96e64ed69e37d0d4bf9d9fd6 (DEFAULT_TABLE table=default.Table_a51a7905e6f848edb48e25b18d0ae196 [id=ba9dbd5eba46433d8166e91188c49599]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.927003 10636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5630f5ea96e64ed69e37d0d4bf9d9fd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.930406 10624 tablet_service.cc:1467] Processing CreateTablet for tablet ca51a5f7e0644df0a793a8be8dbc271e (DEFAULT_TABLE table=default.Table_cd15e5e8e07947459840c3ed5dca1f1d [id=ad71d06959604cdf89d7dbe5716d2d08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.931349 10624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca51a5f7e0644df0a793a8be8dbc271e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.945080 11459 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" instance_seqno: 1790209254919813) as {username='slave'} at 127.10.4.67:40723; Asking this server to re-register.
I20260924 00:21:11.948582 10384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" candidate_term: 3 candidate_status { last_received { term: 2 index: 322 } } ignore_live_leader: false dest_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" is_pre_election: true
I20260924 00:21:11.950558 11412 leader_election.cc:302] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f8bd0b3184f84f5f8da10c9c21472f92; no voters: 1d68651913494ce8afee3c136b2da9cc, b5aa3debe1cd45e19a6bbd0070913ac3
I20260924 00:21:11.954838 11641 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 00:21:11.956285 10909 tablet_service.cc:1467] Processing CreateTablet for tablet 45ba52ea61364b988ce77b265f8c7a79 (DEFAULT_TABLE table=default.Table_a5a83435a7fb47ce8083022c0686b74a [id=a587a50b0c204843a41acacc45c2b150]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.957456 10909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45ba52ea61364b988ce77b265f8c7a79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.962133 11644 leader_election.cc:288] T 6bcfd729e9bd4040b3384ab24164e80d P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:21:11.963653 11644 raft_consensus.cc:2781] T 6bcfd729e9bd4040b3384ab24164e80d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:11.964015 11644 raft_consensus.cc:482] T 6bcfd729e9bd4040b3384ab24164e80d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:11.964366 11644 raft_consensus.cc:3037] T 6bcfd729e9bd4040b3384ab24164e80d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:11.967273 10370 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_a9562c9df88a4874901902e8673278b5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:11.967765 11668 leader_election.cc:288] T 3b8d74761b0b4fd0a22021d3870ec05c P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
W20260924 00:21:11.968642 10370 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a9562c9df88a4874901902e8673278b5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260924 00:21:11.969525 10263 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.10.4.65:33461, user_credentials={real_user=slave}} blocked reactor thread for 64920.4us
I20260924 00:21:11.974861 10908 tablet_service.cc:1467] Processing CreateTablet for tablet e97db32bea1a4cf1bea82d99c8db8147 (DEFAULT_TABLE table=default.Table_8823ba2975054860816e34a4fa104be8 [id=556a0c70dd484c5aaa120eb5f0dfe8e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.975430 10910 tablet_service.cc:1467] Processing CreateTablet for tablet ca51a5f7e0644df0a793a8be8dbc271e (DEFAULT_TABLE table=default.Table_cd15e5e8e07947459840c3ed5dca1f1d [id=ad71d06959604cdf89d7dbe5716d2d08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.975764 10908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e97db32bea1a4cf1bea82d99c8db8147. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.976445 10910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca51a5f7e0644df0a793a8be8dbc271e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:21:11.984992 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 1a7eb76a787a447793fd04686f748d1e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1a7eb76a787a447793fd04686f748d1e already in progress: creating tablet
I20260924 00:21:11.987488 10988 heartbeater.cc:464] Registering TS with master...
I20260924 00:21:11.994632 11459 ts_manager.cc:194] Registered new tserver with Master: c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:12.006369 10920 tablet_service.cc:1467] Processing CreateTablet for tablet 5630f5ea96e64ed69e37d0d4bf9d9fd6 (DEFAULT_TABLE table=default.Table_a51a7905e6f848edb48e25b18d0ae196 [id=ba9dbd5eba46433d8166e91188c49599]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:12.007481 10920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5630f5ea96e64ed69e37d0d4bf9d9fd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:12.007910 11644 raft_consensus.cc:504] T 6bcfd729e9bd4040b3384ab24164e80d P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
W20260924 00:21:12.011919 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet d71ce6053fdd4d53b3e0e596148d4c5c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d71ce6053fdd4d53b3e0e596148d4c5c already in progress: creating tablet
W20260924 00:21:12.012745 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 1a7eb76a787a447793fd04686f748d1e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1a7eb76a787a447793fd04686f748d1e already in progress: creating tablet
I20260924 00:21:12.024693 11041 tablet_bootstrap.cc:492] T 3c850d0474414d2486cd7bbd6568e936 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:12.004887 11456 ts_manager.cc:194] Registered new tserver with Master: b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
W20260924 00:21:12.011619 10876 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:33461, user_credentials={real_user=slave}} blocked reactor thread for 127803us
I20260924 00:21:12.034829 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b8d74761b0b4fd0a22021d3870ec05c" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:21:12.035334 10654 raft_consensus.cc:3037] T 3b8d74761b0b4fd0a22021d3870ec05c P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:12.035291 11644 leader_election.cc:288] T 6bcfd729e9bd4040b3384ab24164e80d P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
W20260924 00:21:12.040898 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet d71ce6053fdd4d53b3e0e596148d4c5c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d71ce6053fdd4d53b3e0e596148d4c5c already in progress: creating tablet
I20260924 00:21:12.047858 10654 raft_consensus.cc:2454] T 3b8d74761b0b4fd0a22021d3870ec05c P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 1.
I20260924 00:21:12.053020 11644 consensus_queue.cc:1035] T 4e9f00abffe14046bf47a6cae7010690 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Connected to new peer: Peer: permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 00:21:12.054515 11724 consensus_queue.cc:1035] T 72d3bb80e1034881b94eb75de5d900c2 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Connected to new peer: Peer: permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 00:21:12.049503 10651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bcfd729e9bd4040b3384ab24164e80d" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:21:12.058565 10651 raft_consensus.cc:3037] T 6bcfd729e9bd4040b3384ab24164e80d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
W20260924 00:21:12.058759 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 1a7eb76a787a447793fd04686f748d1e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1a7eb76a787a447793fd04686f748d1e already in progress: creating tablet
I20260924 00:21:12.066071 11040 raft_consensus.cc:374] T 4e9f00abffe14046bf47a6cae7010690 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:12.066713 11040 raft_consensus.cc:729] T 4e9f00abffe14046bf47a6cae7010690 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:12.067767 11040 consensus_queue.cc:260] T 4e9f00abffe14046bf47a6cae7010690 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:11.911712 10776 tablet_service.cc:1467] Processing CreateTablet for tablet ca51a5f7e0644df0a793a8be8dbc271e (DEFAULT_TABLE table=default.Table_cd15e5e8e07947459840c3ed5dca1f1d [id=ad71d06959604cdf89d7dbe5716d2d08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:11.927901 10779 tablet_service.cc:1467] Processing CreateTablet for tablet e97db32bea1a4cf1bea82d99c8db8147 (DEFAULT_TABLE table=default.Table_8823ba2975054860816e34a4fa104be8 [id=556a0c70dd484c5aaa120eb5f0dfe8e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:12.032506 10800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b8d74761b0b4fd0a22021d3870ec05c" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:21:12.033110 10798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b8d74761b0b4fd0a22021d3870ec05c" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:12.033560 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b43ce1fed0749b8b076b9b026914e07" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:12.034835 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bcfd729e9bd4040b3384ab24164e80d" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:12.070719 10878 leader_election.cc:341] T 3b8d74761b0b4fd0a22021d3870ec05c P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:12.032792 10794 raft_consensus.cc:3037] T 72d3bb80e1034881b94eb75de5d900c2 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
W20260924 00:21:12.072624 10592 leader_election.cc:341] T 5b43ce1fed0749b8b076b9b026914e07 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
W20260924 00:21:12.073307 10878 leader_election.cc:341] T 3b8d74761b0b4fd0a22021d3870ec05c P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
W20260924 00:21:12.074021 10878 leader_election.cc:341] T 6bcfd729e9bd4040b3384ab24164e80d P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:12.035943 10777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e99dc66ce9e41ef98b6b24fa41c1b36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:12.076547 10776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca51a5f7e0644df0a793a8be8dbc271e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:12.037153 10799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bcfd729e9bd4040b3384ab24164e80d" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:21:12.081166 10779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e97db32bea1a4cf1bea82d99c8db8147. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:11.909755 10772 tablet_service.cc:1467] Processing CreateTablet for tablet 5630f5ea96e64ed69e37d0d4bf9d9fd6 (DEFAULT_TABLE table=default.Table_a51a7905e6f848edb48e25b18d0ae196 [id=ba9dbd5eba46433d8166e91188c49599]), partition=RANGE (key) PARTITION UNBOUNDED
W20260924 00:21:12.081885 10878 leader_election.cc:341] T 6bcfd729e9bd4040b3384ab24164e80d P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:12.082096 10772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5630f5ea96e64ed69e37d0d4bf9d9fd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:12.085189 10773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45ba52ea61364b988ce77b265f8c7a79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:12.089174 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22640bf1c2304d978e66c002d74aab51. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:21:12.053081 10589 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:41301, user_credentials={real_user=slave}} blocked reactor thread for 175487us
I20260924 00:21:12.053622 11041 ts_tablet_manager.cc:1397] T 3c850d0474414d2486cd7bbd6568e936 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.450s user 0.010s sys 0.005s
I20260924 00:21:12.093516 11041 raft_consensus.cc:348] T 3c850d0474414d2486cd7bbd6568e936 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:12.094187 11041 raft_consensus.cc:374] T 3c850d0474414d2486cd7bbd6568e936 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:12.094424 11041 raft_consensus.cc:729] T 3c850d0474414d2486cd7bbd6568e936 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:12.094547 10876 leader_election.cc:302] T 3b8d74761b0b4fd0a22021d3870ec05c P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08
I20260924 00:21:12.094920 11041 consensus_queue.cc:260] T 3c850d0474414d2486cd7bbd6568e936 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:12.096288 11724 raft_consensus.cc:482] T 10aa64ed664f40629f5f134052df8f5f P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:12.096681 11724 raft_consensus.cc:504] T 10aa64ed664f40629f5f134052df8f5f P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:12.097818 11041 ts_tablet_manager.cc:1428] T 3c850d0474414d2486cd7bbd6568e936 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260924 00:21:12.098591 11041 tablet_bootstrap.cc:492] T 5107b763acf04df88553ba0af2757570 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:12.103457 10651 raft_consensus.cc:2454] T 6bcfd729e9bd4040b3384ab24164e80d P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 1.
I20260924 00:21:12.104990 11693 raft_consensus.cc:482] T a62b85ebb2b84ae19cad3e3e2dd07a59 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:12.105406 11693 raft_consensus.cc:504] T a62b85ebb2b84ae19cad3e3e2dd07a59 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:12.107231 11693 leader_election.cc:288] T a62b85ebb2b84ae19cad3e3e2dd07a59 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:12.107409 10799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10aa64ed664f40629f5f134052df8f5f" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:12.108879 10878 leader_election.cc:341] T 10aa64ed664f40629f5f134052df8f5f P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:12.108781 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a62b85ebb2b84ae19cad3e3e2dd07a59" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:12.109997 10592 leader_election.cc:341] T a62b85ebb2b84ae19cad3e3e2dd07a59 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:12.110050 11724 leader_election.cc:288] T 10aa64ed664f40629f5f134052df8f5f P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
W20260924 00:21:12.112457 10589 leader_election.cc:341] T 5b43ce1fed0749b8b076b9b026914e07 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:12.114274 10876 leader_election.cc:302] T 6bcfd729e9bd4040b3384ab24164e80d P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08
I20260924 00:21:12.114909 11725 raft_consensus.cc:2781] T 3b8d74761b0b4fd0a22021d3870ec05c P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:12.115324 11725 raft_consensus.cc:686] T 3b8d74761b0b4fd0a22021d3870ec05c P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Becoming Leader. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Running, Role: LEADER
I20260924 00:21:12.115980 11725 consensus_queue.cc:237] T 3b8d74761b0b4fd0a22021d3870ec05c P c8626e300d0644b0908797ad61bd7cef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:12.118136 11668 raft_consensus.cc:2781] T 6bcfd729e9bd4040b3384ab24164e80d P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:12.118517 11668 raft_consensus.cc:686] T 6bcfd729e9bd4040b3384ab24164e80d P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Becoming Leader. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Running, Role: LEADER
I20260924 00:21:12.119228 11668 consensus_queue.cc:237] T 6bcfd729e9bd4040b3384ab24164e80d P c8626e300d0644b0908797ad61bd7cef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:12.112828 10589 leader_election.cc:302] T 5b43ce1fed0749b8b076b9b026914e07 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:12.125095 11693 raft_consensus.cc:2726] T 5b43ce1fed0749b8b076b9b026914e07 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:12.125418 10795 raft_consensus.cc:3037] T 4e9f00abffe14046bf47a6cae7010690 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:12.014515 11039 tablet_bootstrap.cc:654] T 675b9e33edcf4d4ea7fa428b476e3945 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:12.129940 11041 tablet_bootstrap.cc:654] T 5107b763acf04df88553ba0af2757570 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:12.135499 11722 raft_consensus.cc:482] T 675b9e33edcf4d4ea7fa428b476e3945 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:12.136132 11722 raft_consensus.cc:504] T 675b9e33edcf4d4ea7fa428b476e3945 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:12.138031 11722 leader_election.cc:288] T 675b9e33edcf4d4ea7fa428b476e3945 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:12.140221 10799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "675b9e33edcf4d4ea7fa428b476e3945" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:12.141417 10592 leader_election.cc:341] T 675b9e33edcf4d4ea7fa428b476e3945 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:12.143124 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a62b85ebb2b84ae19cad3e3e2dd07a59" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:12.153532 10371 catalog_manager.cc:6040] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: default.Table_76ffb857cf034b6f80598edd90dad953 [id=5fb5e1e90c9440f385e7b04520bcba9c] alter complete (version 1)
W20260924 00:21:12.162830 10372 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.4.65:55561 (request call id 70) took 1245 ms (client timeout 1000 ms). Trace:
W20260924 00:21:12.166628 10373 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.4.66:51189 (request call id 87) took 1117 ms (client timeout 1000 ms). Trace:
W20260924 00:21:12.167054 10373 rpcz_store.cc:269] 0924 00:21:11.048933 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:11.049523 (+ 590us) service_pool.cc:227] Handling call
0924 00:21:11.089042 (+ 39519us) spinlock_profiling.cc:243] Waited 32.8 ms on lock 0x7b54000118d4. stack: 00007fde9ac98033 00007fde9ac97f30 00007fde9a819534 00000000004db2b4 00000000004db249 00007fdea1ba4af0 00007fdea1c27f2a 00007fdea3af3d6c 00007fdea3bedc3e 00007fdea1724fb4 00007fdea1724f42 00007fdea1724e71 00007fdea1724dec 00007fdea17240a2 00007fde9f9ce94c 00007fde9f9cdd86
0924 00:21:12.166569 (+1077527us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":4,"spinlock_wait_cycles":75424128}
I20260924 00:21:12.173919 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59400 (request call id 0) took 3728 ms. Trace:
I20260924 00:21:12.181394 10909 tablet_service.cc:1514] Processing DeleteTablet for tablet b36cbb6520de4df58b88cfa53c20ee7a with delete_type TABLET_DATA_DELETED (replaced by tablet 9e44244181324288ad04afb7096839cb) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:12.182044 11717 tablet_replica.cc:330] T b36cbb6520de4df58b88cfa53c20ee7a P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:12.182832 11717 raft_consensus.cc:2229] T b36cbb6520de4df58b88cfa53c20ee7a P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:12.183269 11717 raft_consensus.cc:2258] T b36cbb6520de4df58b88cfa53c20ee7a P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:12.185035 11717 ts_tablet_manager.cc:1905] T b36cbb6520de4df58b88cfa53c20ee7a P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:12.189414 10634 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b8d74761b0b4fd0a22021d3870ec05c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:07 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:12.189728 10626 tablet_service.cc:1514] Processing DeleteTablet for tablet 675b9e33edcf4d4ea7fa428b476e3945 with delete_type TABLET_DATA_DELETED (replaced by tablet 915e8072851c4d2b90f8af12b1dc58e0) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:12.190095 11719 tablet_replica.cc:330] T 3b8d74761b0b4fd0a22021d3870ec05c P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:12.190311 10630 tablet_service.cc:1514] Processing DeleteTablet for tablet e937cee22326474598cef5c1d09b59c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:07 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:12.190891 11719 raft_consensus.cc:2229] T 3b8d74761b0b4fd0a22021d3870ec05c P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:12.191329 11719 raft_consensus.cc:2258] T 3b8d74761b0b4fd0a22021d3870ec05c P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:12.193706 10627 tablet_service.cc:1514] Processing DeleteTablet for tablet a62b85ebb2b84ae19cad3e3e2dd07a59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:09 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:12.194780 11719 ts_tablet_manager.cc:1905] T 3b8d74761b0b4fd0a22021d3870ec05c P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:12.205029 10367 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 5203b62bde2546e5b99034afde9c0813
I20260924 00:21:12.206353 10367 catalog_manager.cc:6565] ReplaceTablet: tablet 5203b62bde2546e5b99034afde9c0813 deleted and replaced by tablet e68c8ce351bf4f699fddbc0689803ab8
I20260924 00:21:12.207871 10779 tablet_service.cc:1514] Processing DeleteTablet for tablet 60ebab3e93f54383b87aeb54bbde722d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:08 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:12.208510 10776 tablet_service.cc:1514] Processing DeleteTablet for tablet a72742864a6b4299b71757fb308dcadf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:06 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:12.208382 11644 consensus_queue.cc:1035] T 72d3bb80e1034881b94eb75de5d900c2 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Connected to new peer: Peer: permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260924 00:21:12.210671 10371 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.4.67:49569 (request call id 65) took 1158 ms (client timeout 1000 ms). Trace:
I20260924 00:21:12.211763 11040 ts_tablet_manager.cc:1428] T 4e9f00abffe14046bf47a6cae7010690 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.342s user 0.002s sys 0.004s
I20260924 00:21:12.212162 10909 tablet_service.cc:1514] Processing DeleteTablet for tablet 5203b62bde2546e5b99034afde9c0813 with delete_type TABLET_DATA_DELETED (replaced by tablet e68c8ce351bf4f699fddbc0689803ab8) from {username='slave'} at 127.0.0.1:35358
W20260924 00:21:12.163178 10372 rpcz_store.cc:269] 0924 00:21:10.917441 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:10.918256 (+ 815us) service_pool.cc:227] Handling call
0924 00:21:12.162763 (+1244507us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 00:21:11.442990 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 00:21:11.443343 (+ 353us) write_op.cc:432] Acquiring schema lock in shared mode
0924 00:21:11.443364 (+ 21us) write_op.cc:435] Acquired schema lock
0924 00:21:11.443376 (+ 12us) tablet.cc:660] Decoding operations
0924 00:21:11.443607 (+ 231us) write_op.cc:620] Acquiring the partition lock for write op
0924 00:21:11.443643 (+ 36us) write_op.cc:641] Partition lock acquired for write op
0924 00:21:11.443665 (+ 22us) tablet.cc:683] Acquiring locks for 1 operations
0924 00:21:11.469101 (+ 25436us) tablet.cc:699] Row locks acquired
0924 00:21:11.469117 (+ 16us) write_op.cc:260] PREPARE: finished
0924 00:21:11.469223 (+ 106us) write_op.cc:270] Start()
0924 00:21:11.469302 (+ 79us) write_op.cc:276] Timestamp: P: 1790209271469181 usec, L: 0
0924 00:21:11.469375 (+ 73us) op_driver.cc:331] REPLICATION: starting
0924 00:21:11.469814 (+ 439us) log.cc:842] Serialized 356 byte log entry
0924 00:21:11.593289 (+123475us) op_driver.cc:444] REPLICATION: finished
0924 00:21:11.593935 (+ 646us) write_op.cc:301] APPLY: starting
0924 00:21:11.593987 (+ 52us) tablet.cc:1365] starting BulkCheckPresence
0924 00:21:11.594065 (+ 78us) tablet.cc:1368] finished BulkCheckPresence
0924 00:21:11.594147 (+ 82us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 00:21:11.594279 (+ 132us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 00:21:11.594396 (+ 117us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0924 00:21:11.594434 (+ 38us) write_op.cc:312] APPLY: finished
0924 00:21:11.594587 (+ 153us) log.cc:842] Serialized 23 byte log entry
0924 00:21:11.594718 (+ 131us) write_op.cc:489] Releasing partition, row and schema locks
0924 00:21:11.594772 (+ 54us) write_op.cc:454] Released schema lock
0924 00:21:11.594799 (+ 27us) write_op.cc:341] FINISH: Updating metrics
0924 00:21:11.594956 (+ 157us) write_op.cc:489] Releasing partition, row and schema locks
0924 00:21:11.594980 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":514,"apply.run_cpu_time_us":1181,"apply.run_wall_time_us":1180,"num_ops":1,"prepare.queue_time_us":385632,"prepare.run_cpu_time_us":1796,"prepare.run_wall_time_us":28961,"raft.queue_time_us":72,"raft.run_cpu_time_us":506,"raft.run_wall_time_us":505,"replication_time_us":123836}]]}
I20260924 00:21:12.224450 10776 tablet_service.cc:1514] Processing DeleteTablet for tablet 5203b62bde2546e5b99034afde9c0813 with delete_type TABLET_DATA_DELETED (replaced by tablet e68c8ce351bf4f699fddbc0689803ab8) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:12.230762 10634 tablet_service.cc:1514] Processing DeleteTablet for tablet 5203b62bde2546e5b99034afde9c0813 with delete_type TABLET_DATA_DELETED (replaced by tablet e68c8ce351bf4f699fddbc0689803ab8) from {username='slave'} at 127.0.0.1:46896
W20260924 00:21:12.211019 10371 rpcz_store.cc:269] 0924 00:21:11.052568 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:11.052689 (+ 121us) service_pool.cc:227] Handling call
0924 00:21:11.084056 (+ 31367us) spinlock_profiling.cc:243] Waited 29.6 ms on lock 0x7b54000118d4. stack: 00007fde9ac98033 00007fde9ac97f30 00007fde9a819534 00000000004db2b4 00000000004db249 00007fdea1ba4af0 00007fdea1c27f2a 00007fdea3af3d6c 00007fdea3bedc3e 00007fdea1724fb4 00007fdea1724f42 00007fdea1724e71 00007fdea1724dec 00007fdea17240a2 00007fde9f9ce94c 00007fde9f9cdd86
0924 00:21:12.210611 (+1126555us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 00:21:11.599486 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 00:21:11.599832 (+ 346us) write_op.cc:432] Acquiring schema lock in shared mode
0924 00:21:11.599855 (+ 23us) write_op.cc:435] Acquired schema lock
0924 00:21:11.599867 (+ 12us) tablet.cc:660] Decoding operations
0924 00:21:11.600093 (+ 226us) write_op.cc:620] Acquiring the partition lock for write op
0924 00:21:11.600123 (+ 30us) write_op.cc:641] Partition lock acquired for write op
0924 00:21:11.600142 (+ 19us) tablet.cc:683] Acquiring locks for 1 operations
0924 00:21:11.600262 (+ 120us) tablet.cc:699] Row locks acquired
0924 00:21:11.600275 (+ 13us) write_op.cc:260] PREPARE: finished
0924 00:21:11.600373 (+ 98us) write_op.cc:270] Start()
0924 00:21:11.600444 (+ 71us) write_op.cc:276] Timestamp: P: 1790209271600320 usec, L: 0
0924 00:21:11.600458 (+ 14us) op_driver.cc:331] REPLICATION: starting
0924 00:21:11.600911 (+ 453us) log.cc:842] Serialized 527 byte log entry
0924 00:21:11.653244 (+ 52333us) op_driver.cc:444] REPLICATION: finished
0924 00:21:11.672713 (+ 19469us) write_op.cc:301] APPLY: starting
0924 00:21:11.672766 (+ 53us) tablet.cc:1365] starting BulkCheckPresence
0924 00:21:11.672848 (+ 82us) tablet.cc:1368] finished BulkCheckPresence
0924 00:21:11.672967 (+ 119us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 00:21:11.673113 (+ 146us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 00:21:11.673231 (+ 118us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0924 00:21:11.673269 (+ 38us) write_op.cc:312] APPLY: finished
0924 00:21:11.673423 (+ 154us) log.cc:842] Serialized 23 byte log entry
0924 00:21:11.714339 (+ 40916us) write_op.cc:489] Releasing partition, row and schema locks
0924 00:21:11.714405 (+ 66us) write_op.cc:454] Released schema lock
0924 00:21:11.714438 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0924 00:21:11.714612 (+ 174us) write_op.cc:489] Releasing partition, row and schema locks
0924 00:21:11.714634 (+ 22us) write_op.cc:454] Released schema lock
Related trace 'op':
0924 00:21:11.867580 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 00:21:11.867929 (+ 349us) write_op.cc:432] Acquiring schema lock in shared mode
0924 00:21:11.867950 (+ 21us) write_op.cc:435] Acquired schema lock
0924 00:21:11.867963 (+ 13us) tablet.cc:660] Decoding operations
0924 00:21:11.868206 (+ 243us) write_op.cc:620] Acquiring the partition lock for write op
0924 00:21:11.868252 (+ 46us) write_op.cc:641] Partition lock acquired for write op
0924 00:21:11.868274 (+ 22us) tablet.cc:683] Acquiring locks for 1 operations
0924 00:21:11.868402 (+ 128us) tablet.cc:699] Row locks acquired
0924 00:21:11.868415 (+ 13us) write_op.cc:260] PREPARE: finished
0924 00:21:11.868513 (+ 98us) write_op.cc:270] Start()
0924 00:21:11.868586 (+ 73us) write_op.cc:276] Timestamp: P: 1790209271868433 usec, L: 0
0924 00:21:11.868601 (+ 15us) op_driver.cc:331] REPLICATION: starting
0924 00:21:11.889787 (+ 21186us) log.cc:842] Serialized 356 byte log entry
0924 00:21:11.891350 (+ 1563us) spinlock_profiling.cc:243] Waited 20.7 ms on lock 0x7b54000118d4. stack: 00007fde9ac98033 00007fde9ac97f30 00007fde9a819534 00000000004db2b4 00000000004db249 00007fdea1ba4471 00007fdea1c1b198 00007fdea207cade 00007fdea207c2b8 00007fdea2080ba1 00007fdea2080b59 00007fdea2080ae9 00007fdea2080ab1 00007fdea207fdad 0000000000580784 00000000005805b9
0924 00:21:12.146107 (+254757us) op_driver.cc:444] REPLICATION: finished
0924 00:21:12.146990 (+ 883us) write_op.cc:301] APPLY: starting
0924 00:21:12.147042 (+ 52us) tablet.cc:1365] starting BulkCheckPresence
0924 00:21:12.147135 (+ 93us) tablet.cc:1368] finished BulkCheckPresence
0924 00:21:12.147155 (+ 20us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 00:21:12.147357 (+ 202us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 00:21:12.147484 (+ 127us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0924 00:21:12.147529 (+ 45us) write_op.cc:312] APPLY: finished
0924 00:21:12.147698 (+ 169us) log.cc:842] Serialized 23 byte log entry
0924 00:21:12.147854 (+ 156us) write_op.cc:489] Releasing partition, row and schema locks
0924 00:21:12.147908 (+ 54us) write_op.cc:454] Released schema lock
0924 00:21:12.147937 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0924 00:21:12.148115 (+ 178us) write_op.cc:489] Releasing partition, row and schema locks
0924 00:21:12.148138 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":7,"spinlock_wait_cycles":68113792,"child_traces":[["op",{"apply.queue_time_us":19299,"apply.run_cpu_time_us":1382,"apply.run_wall_time_us":42103,"num_ops":1,"prepare.queue_time_us":469986,"prepare.run_cpu_time_us":1627,"prepare.run_wall_time_us":2575,"replication_time_us":52716}],["op",{"apply.queue_time_us":742,"apply.run_cpu_time_us":1296,"apply.run_wall_time_us":1296,"num_ops":1,"prepare.queue_time_us":150756,"prepare.run_cpu_time_us":2355,"prepare.run_wall_time_us":23876,"replication_time_us":277436,"spinlock_wait_cycles":47653120}]]}
I20260924 00:21:12.241925 11717 ts_tablet_manager.cc:1918] T b36cbb6520de4df58b88cfa53c20ee7a P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:12.243556 11717 log.cc:1196] T b36cbb6520de4df58b88cfa53c20ee7a P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/b36cbb6520de4df58b88cfa53c20ee7a
I20260924 00:21:12.244652 11717 ts_tablet_manager.cc:1939] T b36cbb6520de4df58b88cfa53c20ee7a P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:12.246272 11716 tablet_replica.cc:330] T 60ebab3e93f54383b87aeb54bbde722d P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:12.247440 11716 raft_consensus.cc:2229] T 60ebab3e93f54383b87aeb54bbde722d P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:12.247974 10920 tablet_service.cc:1514] Processing DeleteTablet for tablet 59a0a950a8144e99b2ed47be9a567d9a with delete_type TABLET_DATA_DELETED (replaced by tablet d4e120acff3e4c30aed813b0b24fbfee) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:12.253059 10651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10aa64ed664f40629f5f134052df8f5f" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:12.253854 10651 raft_consensus.cc:2454] T 10aa64ed664f40629f5f134052df8f5f P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
I20260924 00:21:12.255430 10876 leader_election.cc:302] T 10aa64ed664f40629f5f134052df8f5f P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08
I20260924 00:21:12.256584 11644 raft_consensus.cc:2781] T 10aa64ed664f40629f5f134052df8f5f P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:12.259160 11716 raft_consensus.cc:2258] T 60ebab3e93f54383b87aeb54bbde722d P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:12.262164 11644 raft_consensus.cc:482] T 10aa64ed664f40629f5f134052df8f5f P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:12.262594 11644 raft_consensus.cc:3037] T 10aa64ed664f40629f5f134052df8f5f P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:12.263013 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet b36cbb6520de4df58b88cfa53c20ee7a (table default.Table_dae0ffc8534047588f6f7cb13a610af5 [id=138be013fbcb4f01a6358d062200fe8f]) successfully deleted
W20260924 00:21:12.265790 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 5203b62bde2546e5b99034afde9c0813 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5203b62bde2546e5b99034afde9c0813 already in progress: creating tablet
I20260924 00:21:12.267324 11717 tablet_replica.cc:330] T 59a0a950a8144e99b2ed47be9a567d9a P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:12.269306 11716 ts_tablet_manager.cc:1905] T 60ebab3e93f54383b87aeb54bbde722d P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:12.269706 11717 raft_consensus.cc:2229] T 59a0a950a8144e99b2ed47be9a567d9a P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:12.272993 11717 raft_consensus.cc:2258] T 59a0a950a8144e99b2ed47be9a567d9a P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus is shut down!
W20260924 00:21:12.273613 10589 leader_election.cc:341] T a62b85ebb2b84ae19cad3e3e2dd07a59 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:12.274068 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "675b9e33edcf4d4ea7fa428b476e3945" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:12.280997 11717 ts_tablet_manager.cc:1905] T 59a0a950a8144e99b2ed47be9a567d9a P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:12.282718 10369 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet fcf651adb1d3459facfe3a8df2a91ed7
I20260924 00:21:12.284036 10369 catalog_manager.cc:6565] ReplaceTablet: tablet fcf651adb1d3459facfe3a8df2a91ed7 deleted and replaced by tablet 86d27e5be8614bef8fa05dc910ff3071
I20260924 00:21:12.284967 10589 leader_election.cc:302] T a62b85ebb2b84ae19cad3e3e2dd07a59 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:12.273749 11040 tablet_bootstrap.cc:492] T cb51d5c6460049108cd7e1974821e578 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:12.286366 10779 tablet_service.cc:1514] Processing DeleteTablet for tablet fcf651adb1d3459facfe3a8df2a91ed7 with delete_type TABLET_DATA_DELETED (replaced by tablet 86d27e5be8614bef8fa05dc910ff3071) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:12.296599 11693 raft_consensus.cc:2726] T a62b85ebb2b84ae19cad3e3e2dd07a59 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:12.285701 10920 tablet_service.cc:1514] Processing DeleteTablet for tablet fcf651adb1d3459facfe3a8df2a91ed7 with delete_type TABLET_DATA_DELETED (replaced by tablet 86d27e5be8614bef8fa05dc910ff3071) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:12.298017 11724 consensus_queue.cc:1035] T 4e9f00abffe14046bf47a6cae7010690 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Connected to new peer: Peer: permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260924 00:21:12.298978 10589 leader_election.cc:341] T 675b9e33edcf4d4ea7fa428b476e3945 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:12.299350 10589 leader_election.cc:302] T 675b9e33edcf4d4ea7fa428b476e3945 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:12.304543 10633 tablet_service.cc:1514] Processing DeleteTablet for tablet fcf651adb1d3459facfe3a8df2a91ed7 with delete_type TABLET_DATA_DELETED (replaced by tablet 86d27e5be8614bef8fa05dc910ff3071) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:12.306478 11644 raft_consensus.cc:504] T 10aa64ed664f40629f5f134052df8f5f P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:12.319324 10372 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_10ca22fe85d94c4b836dc22d24343797" } new_table_name: "default.Table_b7b4d0037f4946659915a6bd5d65b202" modify_external_catalogs: true
I20260924 00:21:12.321810 10373 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_a23f89d2bec746fda3e52abc5a0cea95" } new_table_name: "default.Table_79486769678c44fe98623f5aa06975bb" modify_external_catalogs: true
I20260924 00:21:12.328868 10368 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_60e366f2e65b432da448892cea15e440" } modify_external_catalogs: true
I20260924 00:21:12.329373 10368 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_60e366f2e65b432da448892cea15e440" } modify_external_catalogs: true
I20260924 00:21:12.330205 11040 tablet_bootstrap.cc:654] T cb51d5c6460049108cd7e1974821e578 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:12.335081 10366 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_825c6492de4d43f7ac0521a88ab58441" } modify_external_catalogs: true
I20260924 00:21:12.335556 10366 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_825c6492de4d43f7ac0521a88ab58441" } modify_external_catalogs: true
I20260924 00:21:12.343992 11719 ts_tablet_manager.cc:1918] T 3b8d74761b0b4fd0a22021d3870ec05c P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:12.349892 11039 tablet_bootstrap.cc:492] T 675b9e33edcf4d4ea7fa428b476e3945 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:12.350317 11039 ts_tablet_manager.cc:1397] T 675b9e33edcf4d4ea7fa428b476e3945 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.632s user 0.008s sys 0.001s
I20260924 00:21:12.348656 10367 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet 72d3bb80e1034881b94eb75de5d900c2 (table default.Table_10ca22fe85d94c4b836dc22d24343797 [id=ca942906de0f4019bbdf72ec886fe6fb]). Expected version 2 got 0
I20260924 00:21:12.364158 11040 tablet_bootstrap.cc:492] T cb51d5c6460049108cd7e1974821e578 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:12.364521 11040 ts_tablet_manager.cc:1397] T cb51d5c6460049108cd7e1974821e578 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.152s user 0.005s sys 0.005s
I20260924 00:21:12.361140 11693 raft_consensus.cc:2726] T 675b9e33edcf4d4ea7fa428b476e3945 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:12.369005 10794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10aa64ed664f40629f5f134052df8f5f" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
W20260924 00:21:12.361754 10589 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:41301, user_credentials={real_user=slave}} blocked reactor thread for 62815.8us
I20260924 00:21:12.370095 11719 log.cc:1196] T 3b8d74761b0b4fd0a22021d3870ec05c P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/3b8d74761b0b4fd0a22021d3870ec05c
I20260924 00:21:12.373494 11719 ts_tablet_manager.cc:1939] T 3b8d74761b0b4fd0a22021d3870ec05c P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:12.374418 11039 raft_consensus.cc:348] T 675b9e33edcf4d4ea7fa428b476e3945 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:12.375042 11039 raft_consensus.cc:374] T 675b9e33edcf4d4ea7fa428b476e3945 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:12.375337 11039 raft_consensus.cc:729] T 675b9e33edcf4d4ea7fa428b476e3945 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:12.381959 11719 tablet_replica.cc:330] T 675b9e33edcf4d4ea7fa428b476e3945 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:12.382118 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 3b8d74761b0b4fd0a22021d3870ec05c (table default.Table_0cc93ce7fcfb44089209422666978c62 [id=c0cd199f74494deda9d088345c4c90a3]) successfully deleted
I20260924 00:21:12.374974 11040 raft_consensus.cc:348] T cb51d5c6460049108cd7e1974821e578 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:12.384407 11040 raft_consensus.cc:374] T cb51d5c6460049108cd7e1974821e578 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:12.385289 11040 raft_consensus.cc:729] T cb51d5c6460049108cd7e1974821e578 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:12.386919 10370 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_411ee5e9bc264adab9272f87158db777"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:12.388362 10370 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_411ee5e9bc264adab9272f87158db777 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260924 00:21:12.389189 10878 leader_election.cc:341] T 10aa64ed664f40629f5f134052df8f5f P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:12.389714 11040 consensus_queue.cc:260] T cb51d5c6460049108cd7e1974821e578 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:12.392925 11719 raft_consensus.cc:2229] T 675b9e33edcf4d4ea7fa428b476e3945 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:12.393867 11716 ts_tablet_manager.cc:1918] T 60ebab3e93f54383b87aeb54bbde722d P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:12.394253 11716 log.cc:1196] T 60ebab3e93f54383b87aeb54bbde722d P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/60ebab3e93f54383b87aeb54bbde722d
I20260924 00:21:12.394081 10371 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_9680b062da604b83ad147f325a62ee9c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:12.394874 11716 ts_tablet_manager.cc:1939] T 60ebab3e93f54383b87aeb54bbde722d P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
W20260924 00:21:12.395663 10371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9680b062da604b83ad147f325a62ee9c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:12.398447 11719 raft_consensus.cc:2258] T 675b9e33edcf4d4ea7fa428b476e3945 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:12.400708 11724 raft_consensus.cc:482] T e937cee22326474598cef5c1d09b59c2 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:12.401141 11724 raft_consensus.cc:504] T e937cee22326474598cef5c1d09b59c2 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:12.398030 11040 ts_tablet_manager.cc:1428] T cb51d5c6460049108cd7e1974821e578 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.033s user 0.001s sys 0.006s
I20260924 00:21:12.402921 11724 leader_election.cc:288] T e937cee22326474598cef5c1d09b59c2 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:12.403831 11753 tablet.cc:1722] T bf144ab1a6464a4ab02ac0642c5255e3 P 48fbd603abe146c585d15196e1b5be08: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260924 00:21:12.406147 10652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e937cee22326474598cef5c1d09b59c2" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:12.406769 10652 raft_consensus.cc:2454] T e937cee22326474598cef5c1d09b59c2 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
I20260924 00:21:12.400733 11719 ts_tablet_manager.cc:1905] T 675b9e33edcf4d4ea7fa428b476e3945 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:12.407809 10876 leader_election.cc:302] T e937cee22326474598cef5c1d09b59c2 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:12.408474 11724 raft_consensus.cc:2781] T e937cee22326474598cef5c1d09b59c2 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:12.408788 11724 raft_consensus.cc:482] T e937cee22326474598cef5c1d09b59c2 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:12.409087 11724 raft_consensus.cc:3037] T e937cee22326474598cef5c1d09b59c2 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:12.404732 10794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e937cee22326474598cef5c1d09b59c2" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:12.407330 11040 tablet_bootstrap.cc:492] T 6bcfd729e9bd4040b3384ab24164e80d P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:12.412446 10652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10aa64ed664f40629f5f134052df8f5f" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:21:12.412985 10652 raft_consensus.cc:3037] T 10aa64ed664f40629f5f134052df8f5f P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
W20260924 00:21:12.413753 10878 leader_election.cc:341] T e937cee22326474598cef5c1d09b59c2 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:12.419660 10652 raft_consensus.cc:2454] T 10aa64ed664f40629f5f134052df8f5f P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 1.
I20260924 00:21:12.420617 10876 leader_election.cc:302] T 10aa64ed664f40629f5f134052df8f5f P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08
I20260924 00:21:12.421351 11731 raft_consensus.cc:2781] T 10aa64ed664f40629f5f134052df8f5f P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:12.421746 11731 raft_consensus.cc:686] T 10aa64ed664f40629f5f134052df8f5f P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Becoming Leader. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Running, Role: LEADER
I20260924 00:21:12.421929 11644 leader_election.cc:288] T 10aa64ed664f40629f5f134052df8f5f P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:12.422446 11731 consensus_queue.cc:237] T 10aa64ed664f40629f5f134052df8f5f P c8626e300d0644b0908797ad61bd7cef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:12.427791 11041 tablet_bootstrap.cc:492] T 5107b763acf04df88553ba0af2757570 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:12.428198 11041 ts_tablet_manager.cc:1397] T 5107b763acf04df88553ba0af2757570 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.330s user 0.007s sys 0.002s
I20260924 00:21:12.430279 11041 raft_consensus.cc:348] T 5107b763acf04df88553ba0af2757570 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:12.430904 11041 raft_consensus.cc:374] T 5107b763acf04df88553ba0af2757570 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:12.431135 11041 raft_consensus.cc:729] T 5107b763acf04df88553ba0af2757570 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:12.434239 11724 raft_consensus.cc:504] T e937cee22326474598cef5c1d09b59c2 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:12.435786 11717 ts_tablet_manager.cc:1918] T 59a0a950a8144e99b2ed47be9a567d9a P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:12.438812 11717 log.cc:1196] T 59a0a950a8144e99b2ed47be9a567d9a P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/59a0a950a8144e99b2ed47be9a567d9a
I20260924 00:21:12.440913 10794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e937cee22326474598cef5c1d09b59c2" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
W20260924 00:21:12.442559 10878 leader_election.cc:341] T e937cee22326474598cef5c1d09b59c2 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:12.444017 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 60ebab3e93f54383b87aeb54bbde722d (table default.Table_88117cdd98a74bcfa88b82c3162e7119 [id=ed6b4628a295498da00f778ad37db72e]) successfully deleted
I20260924 00:21:12.445269 11717 ts_tablet_manager.cc:1939] T 59a0a950a8144e99b2ed47be9a567d9a P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:12.451639 11717 tablet_replica.cc:330] T fcf651adb1d3459facfe3a8df2a91ed7 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:12.455976 11717 raft_consensus.cc:2229] T fcf651adb1d3459facfe3a8df2a91ed7 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:21:12.459362 11717 raft_consensus.cc:2258] T fcf651adb1d3459facfe3a8df2a91ed7 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:12.375864 11039 consensus_queue.cc:260] T 675b9e33edcf4d4ea7fa428b476e3945 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:12.463912 11039 ts_tablet_manager.cc:1428] T 675b9e33edcf4d4ea7fa428b476e3945 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.113s user 0.005s sys 0.000s
I20260924 00:21:12.466611 11756 tablet.cc:1722] T bf144ab1a6464a4ab02ac0642c5255e3 P c8626e300d0644b0908797ad61bd7cef: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260924 00:21:12.467674 11039 tablet_bootstrap.cc:492] T 3c850d0474414d2486cd7bbd6568e936 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:12.470425 11717 ts_tablet_manager.cc:1905] T fcf651adb1d3459facfe3a8df2a91ed7 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:12.472631 11039 tablet_bootstrap.cc:654] T 3c850d0474414d2486cd7bbd6568e936 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:12.475782 10656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e937cee22326474598cef5c1d09b59c2" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:21:12.476327 10656 raft_consensus.cc:3037] T e937cee22326474598cef5c1d09b59c2 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:12.477034 10365 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_f24d66ccb67d4a5fb0c34a09c63e089c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:12.480535 11716 tablet_replica.cc:330] T a72742864a6b4299b71757fb308dcadf P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:12.481325 11716 raft_consensus.cc:2229] T a72742864a6b4299b71757fb308dcadf P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:12.481760 11716 raft_consensus.cc:2258] T a72742864a6b4299b71757fb308dcadf P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:12.468345 11041 consensus_queue.cc:260] T 5107b763acf04df88553ba0af2757570 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:12.484207 11716 ts_tablet_manager.cc:1905] T a72742864a6b4299b71757fb308dcadf P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:12.487445 11040 tablet_bootstrap.cc:654] T 6bcfd729e9bd4040b3384ab24164e80d P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:12.496985 10656 raft_consensus.cc:2454] T e937cee22326474598cef5c1d09b59c2 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 1.
I20260924 00:21:12.498118 10876 leader_election.cc:302] T e937cee22326474598cef5c1d09b59c2 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08
I20260924 00:21:12.498875 11731 raft_consensus.cc:2781] T e937cee22326474598cef5c1d09b59c2 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:12.499238 11724 leader_election.cc:288] T e937cee22326474598cef5c1d09b59c2 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:12.510272 11041 ts_tablet_manager.cc:1428] T 5107b763acf04df88553ba0af2757570 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.082s user 0.007s sys 0.001s
I20260924 00:21:12.520998 11040 tablet_bootstrap.cc:492] T 6bcfd729e9bd4040b3384ab24164e80d P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:12.521423 11040 ts_tablet_manager.cc:1397] T 6bcfd729e9bd4040b3384ab24164e80d P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.114s user 0.009s sys 0.007s
I20260924 00:21:12.522848 10370 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_d90e8159c7b34d59bcde6044c2e70bb8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:12.524766 11040 raft_consensus.cc:348] T 6bcfd729e9bd4040b3384ab24164e80d P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:12.525381 11040 raft_consensus.cc:374] T 6bcfd729e9bd4040b3384ab24164e80d P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:12.525614 11040 raft_consensus.cc:729] T 6bcfd729e9bd4040b3384ab24164e80d P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
W20260924 00:21:12.526538 10370 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d90e8159c7b34d59bcde6044c2e70bb8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:12.526153 11040 consensus_queue.cc:260] T 6bcfd729e9bd4040b3384ab24164e80d P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
W20260924 00:21:12.526645 10365 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f24d66ccb67d4a5fb0c34a09c63e089c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:12.450291 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 59a0a950a8144e99b2ed47be9a567d9a (table default.Table_faaa8b08d77d4384a41b2008c9cc98ea [id=3fce8b5a6e82492b8a2355183e566142]) successfully deleted
W20260924 00:21:12.527793 10358 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.10.4.67:41301, user_credentials={real_user=slave}} blocked reactor thread for 77523.2us
I20260924 00:21:12.528076 11040 ts_tablet_manager.cc:1428] T 6bcfd729e9bd4040b3384ab24164e80d P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260924 00:21:12.528808 11040 tablet_bootstrap.cc:492] T b36cbb6520de4df58b88cfa53c20ee7a P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:12.529165 11041 tablet_bootstrap.cc:492] T cf36d53828eb421d87b52b3e4cb1d748 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:12.533821 11040 tablet_bootstrap.cc:654] T b36cbb6520de4df58b88cfa53c20ee7a P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:12.545132 11040 tablet_bootstrap.cc:492] T b36cbb6520de4df58b88cfa53c20ee7a P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:12.545506 11040 ts_tablet_manager.cc:1397] T b36cbb6520de4df58b88cfa53c20ee7a P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.017s user 0.014s sys 0.001s
I20260924 00:21:12.547967 11040 raft_consensus.cc:348] T b36cbb6520de4df58b88cfa53c20ee7a P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:12.548749 11040 raft_consensus.cc:374] T b36cbb6520de4df58b88cfa53c20ee7a P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:12.549028 11040 raft_consensus.cc:729] T b36cbb6520de4df58b88cfa53c20ee7a P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:12.549569 11040 consensus_queue.cc:260] T b36cbb6520de4df58b88cfa53c20ee7a P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:12.551479 11040 ts_tablet_manager.cc:1428] T b36cbb6520de4df58b88cfa53c20ee7a P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260924 00:21:12.552277 11040 tablet_bootstrap.cc:492] T 3b8d74761b0b4fd0a22021d3870ec05c P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:12.499285 11731 raft_consensus.cc:686] T e937cee22326474598cef5c1d09b59c2 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Becoming Leader. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Running, Role: LEADER
I20260924 00:21:12.562858 11731 consensus_queue.cc:237] T e937cee22326474598cef5c1d09b59c2 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:12.518497 11039 tablet_bootstrap.cc:492] T 3c850d0474414d2486cd7bbd6568e936 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:12.565210 11039 ts_tablet_manager.cc:1397] T 3c850d0474414d2486cd7bbd6568e936 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.098s user 0.002s sys 0.012s
I20260924 00:21:12.567950 11039 raft_consensus.cc:348] T 3c850d0474414d2486cd7bbd6568e936 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:12.568567 11039 raft_consensus.cc:374] T 3c850d0474414d2486cd7bbd6568e936 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:12.568783 11039 raft_consensus.cc:729] T 3c850d0474414d2486cd7bbd6568e936 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:12.570817 11039 consensus_queue.cc:260] T 3c850d0474414d2486cd7bbd6568e936 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:12.571295 11758 tablet.cc:1722] T bf144ab1a6464a4ab02ac0642c5255e3 P b10a9082a37b46f09d581944914c1670: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260924 00:21:12.573271 11039 ts_tablet_manager.cc:1428] T 3c850d0474414d2486cd7bbd6568e936 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.008s user 0.004s sys 0.003s
I20260924 00:21:12.574080 11039 tablet_bootstrap.cc:492] T 5b43ce1fed0749b8b076b9b026914e07 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:12.584059 11041 tablet_bootstrap.cc:654] T cf36d53828eb421d87b52b3e4cb1d748 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:12.586028 11039 tablet_bootstrap.cc:654] T 5b43ce1fed0749b8b076b9b026914e07 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:12.586599 11717 ts_tablet_manager.cc:1918] T fcf651adb1d3459facfe3a8df2a91ed7 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:12.591925 11717 log.cc:1196] T fcf651adb1d3459facfe3a8df2a91ed7 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/fcf651adb1d3459facfe3a8df2a91ed7
I20260924 00:21:12.592653 11717 ts_tablet_manager.cc:1939] T fcf651adb1d3459facfe3a8df2a91ed7 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:12.617213 11719 ts_tablet_manager.cc:1918] T 675b9e33edcf4d4ea7fa428b476e3945 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:12.620241 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet fcf651adb1d3459facfe3a8df2a91ed7 (table default.Table_c879109e72984beb85e10878ed02c18f [id=0912091b30374df895fec45a7eafe4cc]) successfully deleted
I20260924 00:21:12.621002 11719 log.cc:1196] T 675b9e33edcf4d4ea7fa428b476e3945 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/675b9e33edcf4d4ea7fa428b476e3945
I20260924 00:21:12.628983 11039 tablet_bootstrap.cc:492] T 5b43ce1fed0749b8b076b9b026914e07 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:12.629349 11039 ts_tablet_manager.cc:1397] T 5b43ce1fed0749b8b076b9b026914e07 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.055s user 0.005s sys 0.003s
I20260924 00:21:12.629710 11719 ts_tablet_manager.cc:1939] T 675b9e33edcf4d4ea7fa428b476e3945 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:12.631299 11719 tablet_replica.cc:330] T e937cee22326474598cef5c1d09b59c2 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:12.632150 11719 raft_consensus.cc:2229] T e937cee22326474598cef5c1d09b59c2 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:12.632675 11719 raft_consensus.cc:2258] T e937cee22326474598cef5c1d09b59c2 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:12.632210 11039 raft_consensus.cc:348] T 5b43ce1fed0749b8b076b9b026914e07 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:12.633447 11039 raft_consensus.cc:374] T 5b43ce1fed0749b8b076b9b026914e07 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:12.633579 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 675b9e33edcf4d4ea7fa428b476e3945 (table default.Table_ee73693b7a6e46a3af53f7828da00f9e [id=9359b05533064c38b9bd589c0bb06471]) successfully deleted
I20260924 00:21:12.633685 11039 raft_consensus.cc:729] T 5b43ce1fed0749b8b076b9b026914e07 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:12.634219 11039 consensus_queue.cc:260] T 5b43ce1fed0749b8b076b9b026914e07 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:12.636124 11039 ts_tablet_manager.cc:1428] T 5b43ce1fed0749b8b076b9b026914e07 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20260924 00:21:12.636871 11039 tablet_bootstrap.cc:492] T cf36d53828eb421d87b52b3e4cb1d748 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:12.640385 10365 master_service.cc:946] ReplaceTablet: received request to replace tablet 4c83e8c5109b43fb8268ec3ed52199be from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:12.647280 11041 tablet_bootstrap.cc:492] T cf36d53828eb421d87b52b3e4cb1d748 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:12.647670 11041 ts_tablet_manager.cc:1397] T cf36d53828eb421d87b52b3e4cb1d748 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.123s user 0.005s sys 0.006s
I20260924 00:21:12.649999 11041 raft_consensus.cc:348] T cf36d53828eb421d87b52b3e4cb1d748 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:12.650741 11041 raft_consensus.cc:374] T cf36d53828eb421d87b52b3e4cb1d748 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:12.651011 11041 raft_consensus.cc:729] T cf36d53828eb421d87b52b3e4cb1d748 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:12.651621 11041 consensus_queue.cc:260] T cf36d53828eb421d87b52b3e4cb1d748 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:12.665275 11041 ts_tablet_manager.cc:1428] T cf36d53828eb421d87b52b3e4cb1d748 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.017s user 0.002s sys 0.003s
I20260924 00:21:12.666368 11041 tablet_bootstrap.cc:492] T 717759eb61674c89b05b8d2b0dc34979 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:12.672508 11039 tablet_bootstrap.cc:654] T cf36d53828eb421d87b52b3e4cb1d748 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:12.685982 11719 ts_tablet_manager.cc:1905] T e937cee22326474598cef5c1d09b59c2 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:12.690900 11041 tablet_bootstrap.cc:654] T 717759eb61674c89b05b8d2b0dc34979 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:12.705552 11766 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b5aa3debe1cd45e19a6bbd0070913ac3)
I20260924 00:21:12.706341 11041 tablet_bootstrap.cc:492] T 717759eb61674c89b05b8d2b0dc34979 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:12.706703 11041 ts_tablet_manager.cc:1397] T 717759eb61674c89b05b8d2b0dc34979 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.040s user 0.008s sys 0.008s
I20260924 00:21:12.708595 11041 raft_consensus.cc:348] T 717759eb61674c89b05b8d2b0dc34979 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:12.709221 11041 raft_consensus.cc:374] T 717759eb61674c89b05b8d2b0dc34979 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:12.709465 11041 raft_consensus.cc:729] T 717759eb61674c89b05b8d2b0dc34979 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:12.706275 11766 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } }
I20260924 00:21:12.711130 11041 consensus_queue.cc:260] T 717759eb61674c89b05b8d2b0dc34979 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:12.720161 10370 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_16819990f8b0403a8947a30c882feb5a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:12.721751 10370 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_16819990f8b0403a8947a30c882feb5a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:12.727142 11719 ts_tablet_manager.cc:1918] T e937cee22326474598cef5c1d09b59c2 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:12.729671 10371 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_396157d80ec44f329a275ccd3167bcb7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:12.731045 10371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_396157d80ec44f329a275ccd3167bcb7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:12.745114 11719 log.cc:1196] T e937cee22326474598cef5c1d09b59c2 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/e937cee22326474598cef5c1d09b59c2
I20260924 00:21:12.745985 11719 ts_tablet_manager.cc:1939] T e937cee22326474598cef5c1d09b59c2 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:12.751673 10370 master_service.cc:946] ReplaceTablet: received request to replace tablet 4c83e8c5109b43fb8268ec3ed52199be from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:12.753407 11041 ts_tablet_manager.cc:1428] T 717759eb61674c89b05b8d2b0dc34979 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.046s user 0.006s sys 0.000s
I20260924 00:21:12.756604 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet e937cee22326474598cef5c1d09b59c2 (table default.Table_d2a991e0c92e4c73b70bfbfb8bea5d58 [id=54d197b284b34dc0a5eea440b0b1e5de]) successfully deleted
I20260924 00:21:12.757468 11719 tablet_replica.cc:330] T a62b85ebb2b84ae19cad3e3e2dd07a59 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:12.757629 11041 tablet_bootstrap.cc:492] T 57ae3d10846a4aaa84eba81b13aafd44 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:12.758345 11719 raft_consensus.cc:2229] T a62b85ebb2b84ae19cad3e3e2dd07a59 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:12.759611 11719 raft_consensus.cc:2258] T a62b85ebb2b84ae19cad3e3e2dd07a59 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:12.763710 11719 ts_tablet_manager.cc:1905] T a62b85ebb2b84ae19cad3e3e2dd07a59 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:12.763458 10371 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_f36e3cefa9f4476594a81b22cf2b3491"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:12.764837 10371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f36e3cefa9f4476594a81b22cf2b3491 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:12.781994 10371 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_ace1c8c32aaa43d6a39b2e3caba246be"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:12.783015 11041 tablet_bootstrap.cc:654] T 57ae3d10846a4aaa84eba81b13aafd44 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
W20260924 00:21:12.783419 10371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ace1c8c32aaa43d6a39b2e3caba246be in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:12.787467 10307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" candidate_term: 3 candidate_status { last_received { term: 2 index: 322 } } ignore_live_leader: false dest_uuid: "1d68651913494ce8afee3c136b2da9cc" is_pre_election: true
I20260924 00:21:12.791306 11040 tablet_bootstrap.cc:654] T 3b8d74761b0b4fd0a22021d3870ec05c P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:12.797418 10384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" candidate_term: 3 candidate_status { last_received { term: 2 index: 322 } } ignore_live_leader: false dest_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" is_pre_election: true
I20260924 00:21:12.798817 11412 leader_election.cc:302] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f8bd0b3184f84f5f8da10c9c21472f92; no voters: 1d68651913494ce8afee3c136b2da9cc, b5aa3debe1cd45e19a6bbd0070913ac3
I20260924 00:21:12.805050 11766 leader_election.cc:288] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1d68651913494ce8afee3c136b2da9cc (127.10.4.126:34479), b5aa3debe1cd45e19a6bbd0070913ac3 (127.10.4.125:34487)
I20260924 00:21:12.809574 11637 raft_consensus.cc:482] T b36cbb6520de4df58b88cfa53c20ee7a P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:12.810194 11637 raft_consensus.cc:504] T b36cbb6520de4df58b88cfa53c20ee7a P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:12.813153 11637 leader_election.cc:288] T b36cbb6520de4df58b88cfa53c20ee7a P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:12.817531 11719 ts_tablet_manager.cc:1918] T a62b85ebb2b84ae19cad3e3e2dd07a59 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:12.822739 11719 log.cc:1196] T a62b85ebb2b84ae19cad3e3e2dd07a59 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/a62b85ebb2b84ae19cad3e3e2dd07a59
I20260924 00:21:12.823513 11719 ts_tablet_manager.cc:1939] T a62b85ebb2b84ae19cad3e3e2dd07a59 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
W20260924 00:21:12.825265 10733 leader_election.cc:341] T b36cbb6520de4df58b88cfa53c20ee7a P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: b36cbb6520de4df58b88cfa53c20ee7a
I20260924 00:21:12.827843 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet a62b85ebb2b84ae19cad3e3e2dd07a59 (table default.Table_3e21dcff4f0b40299e8ddb8ebb7f3c22 [id=3f6c757389bb4213b584f3e5a9ac7ede]) successfully deleted
I20260924 00:21:12.828990 11731 raft_consensus.cc:482] T 675b9e33edcf4d4ea7fa428b476e3945 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:12.829312 11716 ts_tablet_manager.cc:1918] T a72742864a6b4299b71757fb308dcadf P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:12.829411 11731 raft_consensus.cc:504] T 675b9e33edcf4d4ea7fa428b476e3945 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:12.829775 11716 log.cc:1196] T a72742864a6b4299b71757fb308dcadf P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/a72742864a6b4299b71757fb308dcadf
I20260924 00:21:12.830515 11716 ts_tablet_manager.cc:1939] T a72742864a6b4299b71757fb308dcadf P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
W20260924 00:21:12.830605 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 5203b62bde2546e5b99034afde9c0813 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5203b62bde2546e5b99034afde9c0813 already in progress: creating tablet
I20260924 00:21:12.831146 11731 leader_election.cc:288] T 675b9e33edcf4d4ea7fa428b476e3945 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:12.831882 11719 tablet_replica.cc:330] T fcf651adb1d3459facfe3a8df2a91ed7 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:12.832870 10799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "675b9e33edcf4d4ea7fa428b476e3945" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:12.834460 10878 leader_election.cc:341] T 675b9e33edcf4d4ea7fa428b476e3945 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
W20260924 00:21:12.836094 10876 leader_election.cc:341] T 675b9e33edcf4d4ea7fa428b476e3945 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: 675b9e33edcf4d4ea7fa428b476e3945
I20260924 00:21:12.836428 10876 leader_election.cc:302] T 675b9e33edcf4d4ea7fa428b476e3945 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670
I20260924 00:21:12.837333 11731 raft_consensus.cc:2726] T 675b9e33edcf4d4ea7fa428b476e3945 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:12.838863 11719 raft_consensus.cc:2229] T fcf651adb1d3459facfe3a8df2a91ed7 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:12.839330 11719 raft_consensus.cc:2258] T fcf651adb1d3459facfe3a8df2a91ed7 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:12.840917 11766 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 00:21:12.844722 11719 ts_tablet_manager.cc:1905] T fcf651adb1d3459facfe3a8df2a91ed7 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:12.845628 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet a72742864a6b4299b71757fb308dcadf (table default.Table_1c64926dc3894391b7b3776711b0c487 [id=8264957c289442e0b31b70cb7e17f7bc]) successfully deleted
W20260924 00:21:12.847337 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 5203b62bde2546e5b99034afde9c0813 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5203b62bde2546e5b99034afde9c0813 already in progress: creating tablet
I20260924 00:21:12.848227 11716 tablet_replica.cc:330] T fcf651adb1d3459facfe3a8df2a91ed7 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:12.849159 11716 raft_consensus.cc:2229] T fcf651adb1d3459facfe3a8df2a91ed7 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:12.859038 11041 tablet_bootstrap.cc:492] T 57ae3d10846a4aaa84eba81b13aafd44 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:12.859429 11041 ts_tablet_manager.cc:1397] T 57ae3d10846a4aaa84eba81b13aafd44 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.102s user 0.012s sys 0.009s
I20260924 00:21:12.861413 11041 raft_consensus.cc:348] T 57ae3d10846a4aaa84eba81b13aafd44 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:12.862006 11041 raft_consensus.cc:374] T 57ae3d10846a4aaa84eba81b13aafd44 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:12.862231 11041 raft_consensus.cc:729] T 57ae3d10846a4aaa84eba81b13aafd44 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:12.863860 11039 tablet_bootstrap.cc:492] T cf36d53828eb421d87b52b3e4cb1d748 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:12.864444 11039 ts_tablet_manager.cc:1397] T cf36d53828eb421d87b52b3e4cb1d748 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.228s user 0.013s sys 0.007s
I20260924 00:21:12.866494 11039 raft_consensus.cc:348] T cf36d53828eb421d87b52b3e4cb1d748 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:12.867144 11039 raft_consensus.cc:374] T cf36d53828eb421d87b52b3e4cb1d748 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:12.867411 11039 raft_consensus.cc:729] T cf36d53828eb421d87b52b3e4cb1d748 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:12.868036 11039 consensus_queue.cc:260] T cf36d53828eb421d87b52b3e4cb1d748 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:12.869693 11039 ts_tablet_manager.cc:1428] T cf36d53828eb421d87b52b3e4cb1d748 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260924 00:21:12.881078 11041 consensus_queue.cc:260] T 57ae3d10846a4aaa84eba81b13aafd44 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:12.882133 11716 raft_consensus.cc:2258] T fcf651adb1d3459facfe3a8df2a91ed7 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:12.871769 11040 tablet_bootstrap.cc:492] T 3b8d74761b0b4fd0a22021d3870ec05c P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:12.885311 11040 ts_tablet_manager.cc:1397] T 3b8d74761b0b4fd0a22021d3870ec05c P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.333s user 0.005s sys 0.012s
I20260924 00:21:12.887284 11041 ts_tablet_manager.cc:1428] T 57ae3d10846a4aaa84eba81b13aafd44 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.028s user 0.005s sys 0.007s
W20260924 00:21:12.859404 10732 leader_election.cc:341] T b36cbb6520de4df58b88cfa53c20ee7a P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Not found: Tablet not found: b36cbb6520de4df58b88cfa53c20ee7a
I20260924 00:21:12.888051 11041 tablet_bootstrap.cc:492] T 98926dd3850d40938b67779d5fd9a1db P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:12.888059 10732 leader_election.cc:302] T b36cbb6520de4df58b88cfa53c20ee7a P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08; no voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:12.889318 11771 raft_consensus.cc:2726] T b36cbb6520de4df58b88cfa53c20ee7a P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:12.892303 11716 ts_tablet_manager.cc:1905] T fcf651adb1d3459facfe3a8df2a91ed7 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:12.897142 11693 raft_consensus.cc:482] T 3c850d0474414d2486cd7bbd6568e936 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:12.897562 11693 raft_consensus.cc:504] T 3c850d0474414d2486cd7bbd6568e936 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:12.899217 11693 leader_election.cc:288] T 3c850d0474414d2486cd7bbd6568e936 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:12.901340 11041 tablet_bootstrap.cc:654] T 98926dd3850d40938b67779d5fd9a1db P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:12.900664 10799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c850d0474414d2486cd7bbd6568e936" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:12.907883 10592 leader_election.cc:341] T 3c850d0474414d2486cd7bbd6568e936 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:12.909354 11719 ts_tablet_manager.cc:1918] T fcf651adb1d3459facfe3a8df2a91ed7 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:12.901362 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c850d0474414d2486cd7bbd6568e936" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:12.909679 11719 log.cc:1196] T fcf651adb1d3459facfe3a8df2a91ed7 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/fcf651adb1d3459facfe3a8df2a91ed7
I20260924 00:21:12.909897 10942 raft_consensus.cc:2454] T 3c850d0474414d2486cd7bbd6568e936 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b10a9082a37b46f09d581944914c1670 in term 0.
I20260924 00:21:12.910375 11719 ts_tablet_manager.cc:1939] T fcf651adb1d3459facfe3a8df2a91ed7 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:12.911038 10589 leader_election.cc:302] T 3c850d0474414d2486cd7bbd6568e936 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08
I20260924 00:21:12.912101 11693 raft_consensus.cc:2781] T 3c850d0474414d2486cd7bbd6568e936 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:12.912398 11693 raft_consensus.cc:482] T 3c850d0474414d2486cd7bbd6568e936 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:12.912676 11693 raft_consensus.cc:3037] T 3c850d0474414d2486cd7bbd6568e936 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:12.912796 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet fcf651adb1d3459facfe3a8df2a91ed7 (table default.Table_c879109e72984beb85e10878ed02c18f [id=0912091b30374df895fec45a7eafe4cc]) successfully deleted
I20260924 00:21:12.887786 11040 raft_consensus.cc:348] T 3b8d74761b0b4fd0a22021d3870ec05c P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:12.917125 11040 raft_consensus.cc:374] T 3b8d74761b0b4fd0a22021d3870ec05c P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:12.917390 11040 raft_consensus.cc:729] T 3b8d74761b0b4fd0a22021d3870ec05c P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:12.920588 11693 raft_consensus.cc:504] T 3c850d0474414d2486cd7bbd6568e936 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:12.923122 10799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c850d0474414d2486cd7bbd6568e936" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
W20260924 00:21:12.924371 10592 leader_election.cc:341] T 3c850d0474414d2486cd7bbd6568e936 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:12.925210 11040 consensus_queue.cc:260] T 3b8d74761b0b4fd0a22021d3870ec05c P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
W20260924 00:21:12.925928 10359 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet cf0f0f016d074b1fb78d9b427c67fe86 (table default.Table_e412cbac77574c5aafc587eaa698b633 [id=667f4349af3f4eb28461e33f65beeb1c], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet cf0f0f016d074b1fb78d9b427c67fe86 consensus state has no leader
I20260924 00:21:12.926071 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c850d0474414d2486cd7bbd6568e936" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef"
I20260924 00:21:12.926566 10942 raft_consensus.cc:3037] T 3c850d0474414d2486cd7bbd6568e936 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:12.929836 11041 tablet_bootstrap.cc:492] T 98926dd3850d40938b67779d5fd9a1db P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:12.930222 11041 ts_tablet_manager.cc:1397] T 98926dd3850d40938b67779d5fd9a1db P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.042s user 0.014s sys 0.000s
I20260924 00:21:12.932152 11041 raft_consensus.cc:348] T 98926dd3850d40938b67779d5fd9a1db P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:12.932749 11041 raft_consensus.cc:374] T 98926dd3850d40938b67779d5fd9a1db P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:12.932994 11041 raft_consensus.cc:729] T 98926dd3850d40938b67779d5fd9a1db P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:12.933085 10942 raft_consensus.cc:2454] T 3c850d0474414d2486cd7bbd6568e936 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b10a9082a37b46f09d581944914c1670 in term 1.
I20260924 00:21:12.933614 11041 consensus_queue.cc:260] T 98926dd3850d40938b67779d5fd9a1db P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:12.934207 10589 leader_election.cc:302] T 3c850d0474414d2486cd7bbd6568e936 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08
I20260924 00:21:12.934960 11718 raft_consensus.cc:2781] T 3c850d0474414d2486cd7bbd6568e936 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:12.935398 11040 ts_tablet_manager.cc:1428] T 3b8d74761b0b4fd0a22021d3870ec05c P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.050s user 0.005s sys 0.001s
I20260924 00:21:12.939522 11041 ts_tablet_manager.cc:1428] T 98926dd3850d40938b67779d5fd9a1db P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20260924 00:21:12.942557 10371 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_e0eff65c2f3e4800b2345a743d5e1a8a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:12.943981 10371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e0eff65c2f3e4800b2345a743d5e1a8a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:12.949074 11693 leader_election.cc:288] T 3c850d0474414d2486cd7bbd6568e936 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Requested vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:12.953388 11040 tablet_bootstrap.cc:492] T 59a0a950a8144e99b2ed47be9a567d9a P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:12.957516 11771 raft_consensus.cc:482] T cb51d5c6460049108cd7e1974821e578 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:12.957947 11771 raft_consensus.cc:504] T cb51d5c6460049108cd7e1974821e578 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:12.961051 11771 leader_election.cc:288] T cb51d5c6460049108cd7e1974821e578 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
W20260924 00:21:12.961875 10733 leader_election.cc:341] T cb51d5c6460049108cd7e1974821e578 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: cb51d5c6460049108cd7e1974821e578
I20260924 00:21:12.963092 11040 tablet_bootstrap.cc:654] T 59a0a950a8144e99b2ed47be9a567d9a P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:12.963501 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb51d5c6460049108cd7e1974821e578" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:12.964123 10942 raft_consensus.cc:2454] T cb51d5c6460049108cd7e1974821e578 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 0.
I20260924 00:21:12.965241 10732 leader_election.cc:302] T cb51d5c6460049108cd7e1974821e578 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters: b10a9082a37b46f09d581944914c1670
I20260924 00:21:12.966251 11771 raft_consensus.cc:2781] T cb51d5c6460049108cd7e1974821e578 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:12.966581 11771 raft_consensus.cc:482] T cb51d5c6460049108cd7e1974821e578 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:12.966933 11771 raft_consensus.cc:3037] T cb51d5c6460049108cd7e1974821e578 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:12.971097 10371 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_4de2613c5f7243c2bb734ecd585ab408"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:12.972499 10371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4de2613c5f7243c2bb734ecd585ab408 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:12.973348 11039 tablet_bootstrap.cc:492] T 1d5a76bc420b41098d4ebd3dc59aa1fc P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:12.966017 11041 tablet_bootstrap.cc:492] T 9319f9caadb04067a2e9700b5019ed37 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:12.976809 11771 raft_consensus.cc:504] T cb51d5c6460049108cd7e1974821e578 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:12.988533 11039 tablet_bootstrap.cc:654] T 1d5a76bc420b41098d4ebd3dc59aa1fc P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:12.994426 10371 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_b918cc5bd6a94a2a91b42dc02bb68be3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:12.995817 10371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b918cc5bd6a94a2a91b42dc02bb68be3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:13.000435 11771 leader_election.cc:288] T cb51d5c6460049108cd7e1974821e578 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Requested vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
W20260924 00:21:13.001076 10733 leader_election.cc:341] T cb51d5c6460049108cd7e1974821e578 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: cb51d5c6460049108cd7e1974821e578
I20260924 00:21:13.018443 11041 tablet_bootstrap.cc:654] T 9319f9caadb04067a2e9700b5019ed37 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:12.995806 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb51d5c6460049108cd7e1974821e578" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef"
I20260924 00:21:13.018950 10942 raft_consensus.cc:3037] T cb51d5c6460049108cd7e1974821e578 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:13.023849 10371 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_6cebadb2663346f7b016f42a3ed0d1a5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:13.025223 10371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6cebadb2663346f7b016f42a3ed0d1a5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:13.030570 11710 catalog_manager.cc:1259] Loaded cluster ID: 89ba9572944348c7affb30a26e8aad8d
I20260924 00:21:13.030876 11710 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92: loading cluster ID for follower catalog manager: success
I20260924 00:21:13.029508 11718 raft_consensus.cc:686] T 3c850d0474414d2486cd7bbd6568e936 P b10a9082a37b46f09d581944914c1670 [term 1 LEADER]: Becoming Leader. State: Replica: b10a9082a37b46f09d581944914c1670, State: Running, Role: LEADER
I20260924 00:21:13.045076 10942 raft_consensus.cc:2454] T cb51d5c6460049108cd7e1974821e578 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 1.
I20260924 00:21:13.054062 10732 leader_election.cc:302] T cb51d5c6460049108cd7e1974821e578 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters: b10a9082a37b46f09d581944914c1670
I20260924 00:21:13.055337 11771 raft_consensus.cc:2781] T cb51d5c6460049108cd7e1974821e578 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:13.056000 11771 raft_consensus.cc:686] T cb51d5c6460049108cd7e1974821e578 P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: Becoming Leader. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Running, Role: LEADER
I20260924 00:21:13.056813 11771 consensus_queue.cc:237] T cb51d5c6460049108cd7e1974821e578 P 48fbd603abe146c585d15196e1b5be08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:13.059003 11718 consensus_queue.cc:237] T 3c850d0474414d2486cd7bbd6568e936 P b10a9082a37b46f09d581944914c1670 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:13.073197 10799 raft_consensus.cc:3037] T 3b8d74761b0b4fd0a22021d3870ec05c P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:13.073988 11039 tablet_bootstrap.cc:492] T 1d5a76bc420b41098d4ebd3dc59aa1fc P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:13.074361 11039 ts_tablet_manager.cc:1397] T 1d5a76bc420b41098d4ebd3dc59aa1fc P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.101s user 0.009s sys 0.001s
I20260924 00:21:13.076541 11039 raft_consensus.cc:348] T 1d5a76bc420b41098d4ebd3dc59aa1fc P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:13.077164 11039 raft_consensus.cc:374] T 1d5a76bc420b41098d4ebd3dc59aa1fc P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:13.077394 11039 raft_consensus.cc:729] T 1d5a76bc420b41098d4ebd3dc59aa1fc P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:13.077981 11039 consensus_queue.cc:260] T 1d5a76bc420b41098d4ebd3dc59aa1fc P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:13.093115 11724 consensus_queue.cc:1035] T 3b8d74761b0b4fd0a22021d3870ec05c P c8626e300d0644b0908797ad61bd7cef [LEADER]: Connected to new peer: Peer: permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:13.095702 10374 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 72d3bb80e1034881b94eb75de5d900c2 (table default.Table_10ca22fe85d94c4b836dc22d24343797 [id=ca942906de0f4019bbdf72ec886fe6fb]). Expected version 2 got 0
I20260924 00:21:13.096725 10374 catalog_manager.cc:5495] T 4e9f00abffe14046bf47a6cae7010690 P c8626e300d0644b0908797ad61bd7cef reported cstate change: term changed from 0 to 1, leader changed from <none> to c8626e300d0644b0908797ad61bd7cef (127.10.4.67). New cstate: current_term: 1 leader_uuid: "c8626e300d0644b0908797ad61bd7cef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } health_report { overall_health: UNKNOWN } } }
I20260924 00:21:13.097358 10773 tablet_service.cc:1467] Processing CreateTablet for tablet 6fa654977f6442628aba1f478ea86358 (DEFAULT_TABLE table=default.Table_aa8bce70b4eb4f9a957369558580a1a2 [id=95fb80b2ff8044809780c83dcdd07cf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:13.098094 10374 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 4e9f00abffe14046bf47a6cae7010690 (table default.Table_a23f89d2bec746fda3e52abc5a0cea95 [id=423a3ef82b0f488fa8fb082d4e6f63a3]). Expected version 2 got 0
I20260924 00:21:13.098481 10773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fa654977f6442628aba1f478ea86358. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:13.101513 10374 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 6bcfd729e9bd4040b3384ab24164e80d (table default.Table_9670783a92074b639ecb0d6c374811e5 [id=df2a5a74db65431c9ecb750184f64709]). Expected version 1 got 0
I20260924 00:21:13.109954 11039 ts_tablet_manager.cc:1428] T 1d5a76bc420b41098d4ebd3dc59aa1fc P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.035s user 0.005s sys 0.000s
I20260924 00:21:13.110745 11039 tablet_bootstrap.cc:492] T a62b85ebb2b84ae19cad3e3e2dd07a59 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:13.110824 10368 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 1acc16e2d3e14ab092da083b720cc29a
I20260924 00:21:13.113387 10777 tablet_service.cc:1514] Processing DeleteTablet for tablet 1acc16e2d3e14ab092da083b720cc29a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:12 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:13.114089 10368 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:47022 (request call id 378) took 1116 ms. Trace:
I20260924 00:21:13.115899 11716 ts_tablet_manager.cc:1918] T fcf651adb1d3459facfe3a8df2a91ed7 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:13.116250 11716 log.cc:1196] T fcf651adb1d3459facfe3a8df2a91ed7 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/fcf651adb1d3459facfe3a8df2a91ed7
I20260924 00:21:13.114403 10368 rpcz_store.cc:276] 0924 00:21:11.997424 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:12.161056 (+163632us) service_pool.cc:227] Handling call
0924 00:21:12.328783 (+167727us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b54000118d4. stack: 00007fde9ac98033 00007fde9ac97f30 00007fde9a819534 00000000004db2b4 00000000004db249 00007fdea1ba4af0 00007fdea1c27f2a 00007fdea3af3d6c 00007fdea3bef466 00007fdea1730b34 00007fdea1730ac2 00007fdea17309f1 00007fdea173096c 00007fdea172fc22 00007fde9f9ce94c 00007fde9f9cdd86
0924 00:21:12.329773 (+ 990us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0924 00:21:12.330463 (+ 690us) catalog_manager.cc:2752] Modifying in-memory table state
0924 00:21:12.330584 (+ 121us) catalog_manager.cc:2761] Locking tablets
0924 00:21:13.078535 (+747951us) catalog_manager.cc:2775] Removing table and tablets from system table
0924 00:21:13.110464 (+ 31929us) catalog_manager.cc:2793] Removing table from by-name map
0924 00:21:13.110645 (+ 181us) catalog_manager.cc:2809] Committing in-memory state
0924 00:21:13.110743 (+ 98us) catalog_manager.cc:2813] Aborting table tasks
0924 00:21:13.114027 (+ 3284us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 00:21:13.080242 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 00:21:13.080552 (+ 310us) write_op.cc:432] Acquiring schema lock in shared mode
0924 00:21:13.080573 (+ 21us) write_op.cc:435] Acquired schema lock
0924 00:21:13.080586 (+ 13us) tablet.cc:660] Decoding operations
0924 00:21:13.080841 (+ 255us) write_op.cc:620] Acquiring the partition lock for write op
0924 00:21:13.080875 (+ 34us) write_op.cc:641] Partition lock acquired for write op
0924 00:21:13.080896 (+ 21us) tablet.cc:683] Acquiring locks for 2 operations
0924 00:21:13.081169 (+ 273us) tablet.cc:699] Row locks acquired
0924 00:21:13.081182 (+ 13us) write_op.cc:260] PREPARE: finished
0924 00:21:13.081295 (+ 113us) write_op.cc:270] Start()
0924 00:21:13.081366 (+ 71us) write_op.cc:276] Timestamp: P: 1790209273081231 usec, L: 0
0924 00:21:13.081380 (+ 14us) op_driver.cc:331] REPLICATION: starting
0924 00:21:13.081786 (+ 406us) log.cc:842] Serialized 756 byte log entry
0924 00:21:13.106979 (+ 25193us) op_driver.cc:444] REPLICATION: finished
0924 00:21:13.107866 (+ 887us) write_op.cc:301] APPLY: starting
0924 00:21:13.107916 (+ 50us) tablet.cc:1365] starting BulkCheckPresence
0924 00:21:13.108010 (+ 94us) tablet.cc:1368] finished BulkCheckPresence
0924 00:21:13.108091 (+ 81us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 00:21:13.108306 (+ 215us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 00:21:13.108446 (+ 140us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0924 00:21:13.108485 (+ 39us) write_op.cc:312] APPLY: finished
0924 00:21:13.108640 (+ 155us) log.cc:842] Serialized 29 byte log entry
0924 00:21:13.108752 (+ 112us) write_op.cc:489] Releasing partition, row and schema locks
0924 00:21:13.108813 (+ 61us) write_op.cc:454] Released schema lock
0924 00:21:13.108840 (+ 27us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":384227200,"child_traces":[["op",{"apply.queue_time_us":754,"num_ops":2,"prepare.queue_time_us":87,"prepare.run_cpu_time_us":1786,"prepare.run_wall_time_us":8659,"raft.queue_time_us":2089,"raft.run_cpu_time_us":516,"raft.run_wall_time_us":516,"replication_time_us":25532}]]}
I20260924 00:21:13.116935 11716 ts_tablet_manager.cc:1939] T fcf651adb1d3459facfe3a8df2a91ed7 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:13.117532 10906 tablet_service.cc:1514] Processing DeleteTablet for tablet 1acc16e2d3e14ab092da083b720cc29a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:12 UTC) from {username='slave'} at 127.0.0.1:35358
W20260924 00:21:13.119769 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 1acc16e2d3e14ab092da083b720cc29a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1acc16e2d3e14ab092da083b720cc29a already in progress: creating tablet
I20260924 00:21:13.123950 10365 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 4c83e8c5109b43fb8268ec3ed52199be
I20260924 00:21:13.124379 11724 raft_consensus.cc:482] T cf36d53828eb421d87b52b3e4cb1d748 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:13.125155 10365 catalog_manager.cc:6565] ReplaceTablet: tablet 4c83e8c5109b43fb8268ec3ed52199be deleted and replaced by tablet bc23cb60e9e246ff8bc201a3087ffd4d
I20260924 00:21:13.124784 11724 raft_consensus.cc:504] T cf36d53828eb421d87b52b3e4cb1d748 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:13.127425 11724 leader_election.cc:288] T cf36d53828eb421d87b52b3e4cb1d748 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:13.127547 10777 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c83e8c5109b43fb8268ec3ed52199be with delete_type TABLET_DATA_DELETED (replaced by tablet bc23cb60e9e246ff8bc201a3087ffd4d) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:13.128124 11040 tablet_bootstrap.cc:492] T 59a0a950a8144e99b2ed47be9a567d9a P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:13.128571 10906 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c83e8c5109b43fb8268ec3ed52199be with delete_type TABLET_DATA_DELETED (replaced by tablet bc23cb60e9e246ff8bc201a3087ffd4d) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:13.129081 10633 tablet_service.cc:1514] Processing DeleteTablet for tablet 1acc16e2d3e14ab092da083b720cc29a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:12 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:13.129163 10636 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c83e8c5109b43fb8268ec3ed52199be with delete_type TABLET_DATA_DELETED (replaced by tablet bc23cb60e9e246ff8bc201a3087ffd4d) from {username='slave'} at 127.0.0.1:46896
W20260924 00:21:13.131089 10876 consensus_peers.cc:487] T 3b8d74761b0b4fd0a22021d3870ec05c P c8626e300d0644b0908797ad61bd7cef -> Peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Couldn't send request to peer b10a9082a37b46f09d581944914c1670. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3b8d74761b0b4fd0a22021d3870ec05c. This is attempt 1: this message will repeat every 5th retry.
I20260924 00:21:13.131045 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf36d53828eb421d87b52b3e4cb1d748" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:13.131269 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 1acc16e2d3e14ab092da083b720cc29a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1acc16e2d3e14ab092da083b720cc29a already in progress: creating tablet
W20260924 00:21:13.131695 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 4c83e8c5109b43fb8268ec3ed52199be because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4c83e8c5109b43fb8268ec3ed52199be already in progress: creating tablet
I20260924 00:21:13.131917 11040 ts_tablet_manager.cc:1397] T 59a0a950a8144e99b2ed47be9a567d9a P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.179s user 0.014s sys 0.004s
W20260924 00:21:13.132269 10878 leader_election.cc:341] T cf36d53828eb421d87b52b3e4cb1d748 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:13.134009 10652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf36d53828eb421d87b52b3e4cb1d748" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:13.134125 11040 raft_consensus.cc:348] T 59a0a950a8144e99b2ed47be9a567d9a P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:13.134739 11040 raft_consensus.cc:374] T 59a0a950a8144e99b2ed47be9a567d9a P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:13.134611 10652 raft_consensus.cc:2454] T cf36d53828eb421d87b52b3e4cb1d748 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
I20260924 00:21:13.134971 11040 raft_consensus.cc:729] T 59a0a950a8144e99b2ed47be9a567d9a P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
W20260924 00:21:13.136027 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 4c83e8c5109b43fb8268ec3ed52199be because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4c83e8c5109b43fb8268ec3ed52199be already in progress: creating tablet
I20260924 00:21:13.136137 10876 leader_election.cc:302] T cf36d53828eb421d87b52b3e4cb1d748 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08
I20260924 00:21:13.137106 11724 raft_consensus.cc:2781] T cf36d53828eb421d87b52b3e4cb1d748 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:13.137856 11724 raft_consensus.cc:482] T cf36d53828eb421d87b52b3e4cb1d748 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:13.138598 11724 raft_consensus.cc:3037] T cf36d53828eb421d87b52b3e4cb1d748 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:13.140285 11731 raft_consensus.cc:482] T 1d5a76bc420b41098d4ebd3dc59aa1fc P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:13.140657 11731 raft_consensus.cc:504] T 1d5a76bc420b41098d4ebd3dc59aa1fc P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:13.142203 11040 consensus_queue.cc:260] T 59a0a950a8144e99b2ed47be9a567d9a P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:13.143829 10652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d5a76bc420b41098d4ebd3dc59aa1fc" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:13.142362 11731 leader_election.cc:288] T 1d5a76bc420b41098d4ebd3dc59aa1fc P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
W20260924 00:21:13.145408 10876 leader_election.cc:341] T 1d5a76bc420b41098d4ebd3dc59aa1fc P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:13.146284 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d5a76bc420b41098d4ebd3dc59aa1fc" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:13.147382 10878 leader_election.cc:341] T 1d5a76bc420b41098d4ebd3dc59aa1fc P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:13.147704 10878 leader_election.cc:302] T 1d5a76bc420b41098d4ebd3dc59aa1fc P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670
I20260924 00:21:13.148505 11731 raft_consensus.cc:2726] T 1d5a76bc420b41098d4ebd3dc59aa1fc P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:13.149650 11040 ts_tablet_manager.cc:1428] T 59a0a950a8144e99b2ed47be9a567d9a P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.017s user 0.005s sys 0.000s
I20260924 00:21:13.151213 10636 tablet_service.cc:1467] Processing CreateTablet for tablet 6fa654977f6442628aba1f478ea86358 (DEFAULT_TABLE table=default.Table_aa8bce70b4eb4f9a957369558580a1a2 [id=95fb80b2ff8044809780c83dcdd07cf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:13.152132 10636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fa654977f6442628aba1f478ea86358. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:13.154027 10777 tablet_service.cc:1467] Processing CreateTablet for tablet 46f871141e464e4e994cdb8ee85e9e85 (DEFAULT_TABLE table=default.Table_3245e93d2b494073b531fd16a48f1244 [id=a33bb157c42440158194f77c2f9c07a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:13.154448 11041 tablet_bootstrap.cc:492] T 9319f9caadb04067a2e9700b5019ed37 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:13.154862 11041 ts_tablet_manager.cc:1397] T 9319f9caadb04067a2e9700b5019ed37 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.215s user 0.008s sys 0.003s
I20260924 00:21:13.157004 11771 raft_consensus.cc:482] T 3b8d74761b0b4fd0a22021d3870ec05c P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8626e300d0644b0908797ad61bd7cef)
I20260924 00:21:13.157471 11771 raft_consensus.cc:504] T 3b8d74761b0b4fd0a22021d3870ec05c P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:13.159227 11724 raft_consensus.cc:504] T cf36d53828eb421d87b52b3e4cb1d748 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:13.162881 10906 tablet_service.cc:1467] Processing CreateTablet for tablet 6fa654977f6442628aba1f478ea86358 (DEFAULT_TABLE table=default.Table_aa8bce70b4eb4f9a957369558580a1a2 [id=95fb80b2ff8044809780c83dcdd07cf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:13.162999 11041 raft_consensus.cc:348] T 9319f9caadb04067a2e9700b5019ed37 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:13.163605 11041 raft_consensus.cc:374] T 9319f9caadb04067a2e9700b5019ed37 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:13.163883 10906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fa654977f6442628aba1f478ea86358. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:13.164243 10633 tablet_service.cc:1467] Processing CreateTablet for tablet 46f871141e464e4e994cdb8ee85e9e85 (DEFAULT_TABLE table=default.Table_3245e93d2b494073b531fd16a48f1244 [id=a33bb157c42440158194f77c2f9c07a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:13.165143 10633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46f871141e464e4e994cdb8ee85e9e85. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:21:13.167392 10733 leader_election.cc:341] T 3b8d74761b0b4fd0a22021d3870ec05c P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: 3b8d74761b0b4fd0a22021d3870ec05c
I20260924 00:21:13.169062 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b8d74761b0b4fd0a22021d3870ec05c" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:13.169454 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet fcf651adb1d3459facfe3a8df2a91ed7 (table default.Table_c879109e72984beb85e10878ed02c18f [id=0912091b30374df895fec45a7eafe4cc]) successfully deleted
I20260924 00:21:13.170183 10732 leader_election.cc:302] T 3b8d74761b0b4fd0a22021d3870ec05c P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08; no voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:13.170986 11751 raft_consensus.cc:2726] T 3b8d74761b0b4fd0a22021d3870ec05c P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 00:21:13.163864 11041 raft_consensus.cc:729] T 9319f9caadb04067a2e9700b5019ed37 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:13.172204 11041 consensus_queue.cc:260] T 9319f9caadb04067a2e9700b5019ed37 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:13.172914 11771 leader_election.cc:288] T 3b8d74761b0b4fd0a22021d3870ec05c P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:13.176440 11041 ts_tablet_manager.cc:1428] T 9319f9caadb04067a2e9700b5019ed37 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.021s user 0.005s sys 0.000s
I20260924 00:21:13.177635 11041 tablet_bootstrap.cc:492] T 4c2a729a802e4abe9ab4f989a40198be P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:13.180446 11724 leader_election.cc:288] T cf36d53828eb421d87b52b3e4cb1d748 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:13.180294 10652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf36d53828eb421d87b52b3e4cb1d748" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:21:13.181212 10652 raft_consensus.cc:3037] T cf36d53828eb421d87b52b3e4cb1d748 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:13.182435 11039 tablet_bootstrap.cc:654] T a62b85ebb2b84ae19cad3e3e2dd07a59 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:13.187068 10652 raft_consensus.cc:2454] T cf36d53828eb421d87b52b3e4cb1d748 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 1.
I20260924 00:21:13.189534 10876 leader_election.cc:302] T cf36d53828eb421d87b52b3e4cb1d748 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:13.190557 11724 raft_consensus.cc:2781] T cf36d53828eb421d87b52b3e4cb1d748 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:13.191107 11724 raft_consensus.cc:686] T cf36d53828eb421d87b52b3e4cb1d748 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Becoming Leader. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Running, Role: LEADER
I20260924 00:21:13.191885 11724 consensus_queue.cc:237] T cf36d53828eb421d87b52b3e4cb1d748 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:13.204712 10772 tablet_service.cc:1467] Processing CreateTablet for tablet e3dbe5c4827344ed8fe19ae826f62709 (DEFAULT_TABLE table=default.Table_9e4907fea442455b9ac9c793dfabf496 [id=41a14bd521754a1e8fdd166c7a83bd62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:13.207008 10368 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_b7b77b7460bf4db393567227b3536333"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:13.208654 10366 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 8d34535febc94548b48b6b73e1ba85e0
I20260924 00:21:13.210494 10366 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:47022 (request call id 381) took 1142 ms. Trace:
I20260924 00:21:13.198956 10920 tablet_service.cc:1467] Processing CreateTablet for tablet 46f871141e464e4e994cdb8ee85e9e85 (DEFAULT_TABLE table=default.Table_3245e93d2b494073b531fd16a48f1244 [id=a33bb157c42440158194f77c2f9c07a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:13.211354 10920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46f871141e464e4e994cdb8ee85e9e85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:13.213419 10624 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d34535febc94548b48b6b73e1ba85e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:12 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:13.213853 10774 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d34535febc94548b48b6b73e1ba85e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:12 UTC) from {username='slave'} at 127.0.0.1:35996
W20260924 00:21:13.214807 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 8d34535febc94548b48b6b73e1ba85e0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8d34535febc94548b48b6b73e1ba85e0 already in progress: creating tablet
I20260924 00:21:13.216024 10371 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_36c8c183203d418c9a15405725842b64"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:13.216400 10365 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_a85971df84464ddd9ca760e1f0e2ec3f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:13.210764 10366 rpcz_store.cc:276] 0924 00:21:12.068312 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:12.225577 (+157265us) service_pool.cc:227] Handling call
0924 00:21:12.334996 (+109419us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b54000118d4. stack: 00007fde9ac98033 00007fde9ac97f30 00007fde9a819534 00000000004db2b4 00000000004db249 00007fdea1ba4af0 00007fdea1c27f2a 00007fdea3af3d6c 00007fdea3bef466 00007fdea1730b34 00007fdea1730ac2 00007fdea17309f1 00007fdea173096c 00007fdea172fc22 00007fde9f9ce94c 00007fde9f9cdd86
0924 00:21:12.335927 (+ 931us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0924 00:21:12.336408 (+ 481us) catalog_manager.cc:2752] Modifying in-memory table state
0924 00:21:12.336513 (+ 105us) catalog_manager.cc:2761] Locking tablets
0924 00:21:13.093886 (+757373us) catalog_manager.cc:2775] Removing table and tablets from system table
0924 00:21:13.208271 (+114385us) catalog_manager.cc:2793] Removing table from by-name map
0924 00:21:13.208471 (+ 200us) catalog_manager.cc:2809] Committing in-memory state
0924 00:21:13.208565 (+ 94us) catalog_manager.cc:2813] Aborting table tasks
0924 00:21:13.210444 (+ 1879us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 00:21:13.131410 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 00:21:13.146426 (+ 15016us) write_op.cc:432] Acquiring schema lock in shared mode
0924 00:21:13.146450 (+ 24us) write_op.cc:435] Acquired schema lock
0924 00:21:13.146462 (+ 12us) tablet.cc:660] Decoding operations
0924 00:21:13.146751 (+ 289us) write_op.cc:620] Acquiring the partition lock for write op
0924 00:21:13.146787 (+ 36us) write_op.cc:641] Partition lock acquired for write op
0924 00:21:13.146808 (+ 21us) tablet.cc:683] Acquiring locks for 2 operations
0924 00:21:13.147043 (+ 235us) tablet.cc:699] Row locks acquired
0924 00:21:13.147055 (+ 12us) write_op.cc:260] PREPARE: finished
0924 00:21:13.147152 (+ 97us) write_op.cc:270] Start()
0924 00:21:13.147228 (+ 76us) write_op.cc:276] Timestamp: P: 1790209273147095 usec, L: 0
0924 00:21:13.147243 (+ 15us) op_driver.cc:331] REPLICATION: starting
0924 00:21:13.147722 (+ 479us) log.cc:842] Serialized 725 byte log entry
0924 00:21:13.205934 (+ 58212us) op_driver.cc:444] REPLICATION: finished
0924 00:21:13.207053 (+ 1119us) write_op.cc:301] APPLY: starting
0924 00:21:13.207103 (+ 50us) tablet.cc:1365] starting BulkCheckPresence
0924 00:21:13.207203 (+ 100us) tablet.cc:1368] finished BulkCheckPresence
0924 00:21:13.207286 (+ 83us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 00:21:13.207497 (+ 211us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 00:21:13.207623 (+ 126us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0924 00:21:13.207667 (+ 44us) write_op.cc:312] APPLY: finished
0924 00:21:13.207829 (+ 162us) log.cc:842] Serialized 29 byte log entry
0924 00:21:13.207978 (+ 149us) write_op.cc:489] Releasing partition, row and schema locks
0924 00:21:13.208050 (+ 72us) write_op.cc:454] Released schema lock
0924 00:21:13.208082 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0924 00:21:13.209143 (+ 1061us) write_op.cc:489] Releasing partition, row and schema locks
0924 00:21:13.209169 (+ 26us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":249606528,"child_traces":[["op",{"apply.queue_time_us":975,"apply.run_cpu_time_us":1491,"apply.run_wall_time_us":2338,"num_ops":2,"prepare.queue_time_us":35866,"prepare.run_cpu_time_us":1980,"prepare.run_wall_time_us":28481,"raft.queue_time_us":2076,"raft.run_cpu_time_us":502,"raft.run_wall_time_us":502,"replication_time_us":58611}]]}
W20260924 00:21:13.218107 10371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_36c8c183203d418c9a15405725842b64 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260924 00:21:13.217779 10365 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a85971df84464ddd9ca760e1f0e2ec3f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:13.223587 11041 tablet_bootstrap.cc:654] T 4c2a729a802e4abe9ab4f989a40198be P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:13.224524 10915 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d34535febc94548b48b6b73e1ba85e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:12 UTC) from {username='slave'} at 127.0.0.1:35358
W20260924 00:21:13.225845 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 8d34535febc94548b48b6b73e1ba85e0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8d34535febc94548b48b6b73e1ba85e0 already in progress: creating tablet
W20260924 00:21:13.227540 10368 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b7b77b7460bf4db393567227b3536333 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:13.231207 11039 tablet_bootstrap.cc:492] T a62b85ebb2b84ae19cad3e3e2dd07a59 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:13.231339 10369 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 72d3bb80e1034881b94eb75de5d900c2 (table default.Table_10ca22fe85d94c4b836dc22d24343797 [id=ca942906de0f4019bbdf72ec886fe6fb]). Expected version 2 got 0
I20260924 00:21:13.231627 11039 ts_tablet_manager.cc:1397] T a62b85ebb2b84ae19cad3e3e2dd07a59 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.121s user 0.007s sys 0.001s
I20260924 00:21:13.231859 10369 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 4e9f00abffe14046bf47a6cae7010690 (table default.Table_a23f89d2bec746fda3e52abc5a0cea95 [id=423a3ef82b0f488fa8fb082d4e6f63a3]). Expected version 2 got 0
I20260924 00:21:13.237872 10370 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 72d3bb80e1034881b94eb75de5d900c2 (table default.Table_10ca22fe85d94c4b836dc22d24343797 [id=ca942906de0f4019bbdf72ec886fe6fb]). Expected version 2 got 0
I20260924 00:21:13.239205 10370 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 4e9f00abffe14046bf47a6cae7010690 (table default.Table_a23f89d2bec746fda3e52abc5a0cea95 [id=423a3ef82b0f488fa8fb082d4e6f63a3]). Expected version 2 got 0
I20260924 00:21:13.239960 10370 catalog_manager.cc:5495] T 10aa64ed664f40629f5f134052df8f5f P c8626e300d0644b0908797ad61bd7cef reported cstate change: term changed from 0 to 1, leader changed from <none> to c8626e300d0644b0908797ad61bd7cef (127.10.4.67). New cstate: current_term: 1 leader_uuid: "c8626e300d0644b0908797ad61bd7cef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } health_report { overall_health: UNKNOWN } } }
I20260924 00:21:13.249451 11718 raft_consensus.cc:482] T 9319f9caadb04067a2e9700b5019ed37 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:13.250178 10624 tablet_service.cc:1467] Processing CreateTablet for tablet e3dbe5c4827344ed8fe19ae826f62709 (DEFAULT_TABLE table=default.Table_9e4907fea442455b9ac9c793dfabf496 [id=41a14bd521754a1e8fdd166c7a83bd62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:13.249929 11718 raft_consensus.cc:504] T 9319f9caadb04067a2e9700b5019ed37 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:13.251317 10624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3dbe5c4827344ed8fe19ae826f62709. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:13.254390 11718 leader_election.cc:288] T 9319f9caadb04067a2e9700b5019ed37 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:13.254274 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9319f9caadb04067a2e9700b5019ed37" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
W20260924 00:21:13.256577 10589 leader_election.cc:341] T 9319f9caadb04067a2e9700b5019ed37 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:13.256516 10915 tablet_service.cc:1467] Processing CreateTablet for tablet e3dbe5c4827344ed8fe19ae826f62709 (DEFAULT_TABLE table=default.Table_9e4907fea442455b9ac9c793dfabf496 [id=41a14bd521754a1e8fdd166c7a83bd62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:13.257455 10915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3dbe5c4827344ed8fe19ae826f62709. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:13.245211 11039 raft_consensus.cc:348] T a62b85ebb2b84ae19cad3e3e2dd07a59 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:13.257865 11039 raft_consensus.cc:374] T a62b85ebb2b84ae19cad3e3e2dd07a59 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:13.258083 11039 raft_consensus.cc:729] T a62b85ebb2b84ae19cad3e3e2dd07a59 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:13.266472 10794 raft_consensus.cc:3037] T 6bcfd729e9bd4040b3384ab24164e80d P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:13.266743 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9319f9caadb04067a2e9700b5019ed37" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:13.268249 10592 leader_election.cc:341] T 9319f9caadb04067a2e9700b5019ed37 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:13.268625 10592 leader_election.cc:302] T 9319f9caadb04067a2e9700b5019ed37 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:13.270519 11776 raft_consensus.cc:2726] T 9319f9caadb04067a2e9700b5019ed37 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:13.271824 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf36d53828eb421d87b52b3e4cb1d748" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
W20260924 00:21:13.273146 10878 leader_election.cc:341] T cf36d53828eb421d87b52b3e4cb1d748 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:13.275275 11776 raft_consensus.cc:482] T 5107b763acf04df88553ba0af2757570 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:13.275665 11776 raft_consensus.cc:504] T 5107b763acf04df88553ba0af2757570 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:13.277343 11041 tablet_bootstrap.cc:492] T 4c2a729a802e4abe9ab4f989a40198be P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:13.277700 11041 ts_tablet_manager.cc:1397] T 4c2a729a802e4abe9ab4f989a40198be P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.100s user 0.006s sys 0.006s
I20260924 00:21:13.277767 11776 leader_election.cc:288] T 5107b763acf04df88553ba0af2757570 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:13.278465 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5107b763acf04df88553ba0af2757570" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:13.279174 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5107b763acf04df88553ba0af2757570" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:13.280136 10589 leader_election.cc:341] T 5107b763acf04df88553ba0af2757570 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:13.281471 11731 consensus_queue.cc:1035] T 6bcfd729e9bd4040b3384ab24164e80d P c8626e300d0644b0908797ad61bd7cef [LEADER]: Connected to new peer: Peer: permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:13.283280 10777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46f871141e464e4e994cdb8ee85e9e85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:13.285593 10772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3dbe5c4827344ed8fe19ae826f62709. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:13.251876 10370 catalog_manager.cc:5495] T 6bcfd729e9bd4040b3384ab24164e80d P c8626e300d0644b0908797ad61bd7cef reported cstate change: term changed from 0 to 1, leader changed from <none> to c8626e300d0644b0908797ad61bd7cef (127.10.4.67). New cstate: current_term: 1 leader_uuid: "c8626e300d0644b0908797ad61bd7cef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } health_report { overall_health: HEALTHY } } }
I20260924 00:21:13.288447 10370 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 6bcfd729e9bd4040b3384ab24164e80d (table default.Table_9670783a92074b639ecb0d6c374811e5 [id=df2a5a74db65431c9ecb750184f64709]). Expected version 1 got 0
I20260924 00:21:13.289868 10370 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 1d5a76bc420b41098d4ebd3dc59aa1fc (table default.Table_efa7000cc59a43639d964cd555a08961 [id=ff77e539ecb44767bb76cf076799a4c2]). Expected version 1 got 0
W20260924 00:21:13.290421 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 1acc16e2d3e14ab092da083b720cc29a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1acc16e2d3e14ab092da083b720cc29a already in progress: creating tablet
W20260924 00:21:13.290982 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 4c83e8c5109b43fb8268ec3ed52199be because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4c83e8c5109b43fb8268ec3ed52199be already in progress: creating tablet
W20260924 00:21:13.291477 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 8d34535febc94548b48b6b73e1ba85e0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8d34535febc94548b48b6b73e1ba85e0 already in progress: creating tablet
W20260924 00:21:13.293545 10592 leader_election.cc:341] T 5107b763acf04df88553ba0af2757570 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:13.293906 10592 leader_election.cc:302] T 5107b763acf04df88553ba0af2757570 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:13.295459 10372 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:47022 (request call id 380) took 1230 ms. Trace:
I20260924 00:21:13.296254 11041 raft_consensus.cc:348] T 4c2a729a802e4abe9ab4f989a40198be P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:13.298606 10912 tablet_service.cc:1467] Processing CreateTablet for tablet 73018242c4f842e59160eb188ede78dc (DEFAULT_TABLE table=default.Table_acfad28894184501be9e3f9e38fea952 [id=cdcf0facca9849d990e8e9463f288410]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:13.298844 11041 raft_consensus.cc:374] T 4c2a729a802e4abe9ab4f989a40198be P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260924 00:21:13.298983 10367 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.4.66:51189 (request call id 88) took 1117 ms (client timeout 1000 ms). Trace:
I20260924 00:21:13.299086 11041 raft_consensus.cc:729] T 4c2a729a802e4abe9ab4f989a40198be P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:13.299593 10912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73018242c4f842e59160eb188ede78dc. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:21:13.299280 10367 rpcz_store.cc:269] 0924 00:21:12.181894 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:12.206970 (+ 25076us) service_pool.cc:227] Handling call
0924 00:21:12.332242 (+125272us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b54000118d4. stack: 00007fde9ac98033 00007fde9ac97f30 00007fde9a819534 00000000004db2b4 00000000004db249 00007fdea1ba4af0 00007fdea1c27f2a 00007fdea3af3d6c 00007fdea3bedc3e 00007fdea1724fb4 00007fdea1724f42 00007fdea1724e71 00007fdea1724dec 00007fdea17240a2 00007fde9f9ce94c 00007fde9f9cdd86
0924 00:21:13.298920 (+966678us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":7,"spinlock_wait_cycles":256305024}
I20260924 00:21:13.299952 11041 consensus_queue.cc:260] T 4c2a729a802e4abe9ab4f989a40198be P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:13.301160 11776 raft_consensus.cc:2726] T 5107b763acf04df88553ba0af2757570 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:13.302990 11041 ts_tablet_manager.cc:1428] T 4c2a729a802e4abe9ab4f989a40198be P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.025s user 0.005s sys 0.001s
W20260924 00:21:13.306964 10369 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.4.65:55561 (request call id 72) took 1019 ms (client timeout 1000 ms). Trace:
W20260924 00:21:13.307327 10369 rpcz_store.cc:269] 0924 00:21:12.287080 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:12.287972 (+ 892us) service_pool.cc:227] Handling call
0924 00:21:12.337033 (+ 49061us) spinlock_profiling.cc:243] Waited 45.1 ms on lock 0x7b54000118d4. stack: 00007fde9ac98033 00007fde9ac97f30 00007fde9a819534 00000000004db2b4 00000000004db249 00007fdea1ba4af0 00007fdea1c27f2a 00007fdea3af3d6c 00007fdea3bedc3e 00007fdea1724fb4 00007fdea1724f42 00007fdea1724e71 00007fdea1724dec 00007fdea17240a2 00007fde9f9ce94c 00007fde9f9cdd86
0924 00:21:13.306894 (+969861us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":3,"spinlock_wait_cycles":103703424}
I20260924 00:21:13.308789 10908 tablet_service.cc:1467] Processing CreateTablet for tablet 1cc672cf88554772bc4470d6120db6f6 (DEFAULT_TABLE table=default.Table_f1ae1e8ac652481680491d0da3355484 [id=36d61d1504cf495a9643bad809812d43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:13.309830 10908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1cc672cf88554772bc4470d6120db6f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:13.311403 10774 tablet_service.cc:1467] Processing CreateTablet for tablet 1cc672cf88554772bc4470d6120db6f6 (DEFAULT_TABLE table=default.Table_f1ae1e8ac652481680491d0da3355484 [id=36d61d1504cf495a9643bad809812d43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:13.311640 10779 tablet_service.cc:1514] Processing DeleteTablet for tablet a72742864a6b4299b71757fb308dcadf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:06 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:13.313372 10776 tablet_service.cc:1514] Processing DeleteTablet for tablet 60ebab3e93f54383b87aeb54bbde722d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:08 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:13.314110 10775 tablet_service.cc:1514] Processing DeleteTablet for tablet a0f71049f6b3478da3b07ebc9293ce64 with delete_type TABLET_DATA_DELETED (replaced by tablet d574642ebc6340fcb5e83c36804ee9f8) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:13.314769 10780 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c583d9ba1414f8ea2e605b4b6b076f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:01 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:13.316591 10373 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:47022 (request call id 379) took 1253 ms. Trace:
I20260924 00:21:13.322722 10773 tablet_service.cc:1467] Processing CreateTablet for tablet 73018242c4f842e59160eb188ede78dc (DEFAULT_TABLE table=default.Table_acfad28894184501be9e3f9e38fea952 [id=cdcf0facca9849d990e8e9463f288410]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:13.295723 10372 rpcz_store.cc:276] 0924 00:21:12.064726 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:12.216223 (+151497us) service_pool.cc:227] Handling call
0924 00:21:12.319231 (+103008us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b54000118d4. stack: 00007fde9ac98033 00007fde9ac97f30 00007fde9a819534 00000000004db2b4 00000000004db249 00007fdea1ba4af0 00007fdea1c27f2a 00007fdea3af3d6c 00007fdea3bef7d6 00007fdea1735634 00007fdea17355c2 00007fdea17354f1 00007fdea173546c 00007fdea1734722 00007fde9f9ce94c 00007fde9f9cdd86
0924 00:21:12.319882 (+ 651us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0924 00:21:12.320516 (+ 634us) catalog_manager.cc:3616] Apply alter schema
0924 00:21:12.341343 (+ 20827us) catalog_manager.cc:3639] Acquired catalog manager lock
0924 00:21:12.341516 (+ 173us) catalog_manager.cc:3786] Updating metadata on disk
0924 00:21:12.513121 (+171605us) catalog_manager.cc:3817] Committing alterations to in-memory state
0924 00:21:13.295401 (+782280us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 00:21:12.342706 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 00:21:12.343043 (+ 337us) write_op.cc:432] Acquiring schema lock in shared mode
0924 00:21:12.343063 (+ 20us) write_op.cc:435] Acquired schema lock
0924 00:21:12.343075 (+ 12us) tablet.cc:660] Decoding operations
0924 00:21:12.343291 (+ 216us) write_op.cc:620] Acquiring the partition lock for write op
0924 00:21:12.343341 (+ 50us) write_op.cc:641] Partition lock acquired for write op
0924 00:21:12.343361 (+ 20us) tablet.cc:683] Acquiring locks for 1 operations
0924 00:21:12.343484 (+ 123us) tablet.cc:699] Row locks acquired
0924 00:21:12.343496 (+ 12us) write_op.cc:260] PREPARE: finished
0924 00:21:12.343587 (+ 91us) write_op.cc:270] Start()
0924 00:21:12.343662 (+ 75us) write_op.cc:276] Timestamp: P: 1790209272343510 usec, L: 0
0924 00:21:12.343677 (+ 15us) op_driver.cc:331] REPLICATION: starting
0924 00:21:12.344087 (+ 410us) log.cc:842] Serialized 405 byte log entry
0924 00:21:12.509469 (+165382us) op_driver.cc:444] REPLICATION: finished
0924 00:21:12.510096 (+ 627us) write_op.cc:301] APPLY: starting
0924 00:21:12.510176 (+ 80us) tablet.cc:1365] starting BulkCheckPresence
0924 00:21:12.510263 (+ 87us) tablet.cc:1368] finished BulkCheckPresence
0924 00:21:12.510360 (+ 97us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 00:21:12.510504 (+ 144us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 00:21:12.510622 (+ 118us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0924 00:21:12.510663 (+ 41us) write_op.cc:312] APPLY: finished
0924 00:21:12.510840 (+ 177us) log.cc:842] Serialized 23 byte log entry
0924 00:21:12.510977 (+ 137us) write_op.cc:489] Releasing partition, row and schema locks
0924 00:21:12.511034 (+ 57us) write_op.cc:454] Released schema lock
0924 00:21:12.511060 (+ 26us) write_op.cc:341] FINISH: Updating metrics
0924 00:21:12.511221 (+ 161us) write_op.cc:489] Releasing partition, row and schema locks
0924 00:21:12.511243 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":233654144,"child_traces":[["op",{"apply.queue_time_us":488,"apply.run_cpu_time_us":1297,"apply.run_wall_time_us":1296,"num_ops":1,"prepare.queue_time_us":95,"prepare.run_cpu_time_us":1548,"prepare.run_wall_time_us":1549,"replication_time_us":165718}]]}
I20260924 00:21:13.327584 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1cc672cf88554772bc4470d6120db6f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:13.335287 11040 tablet_bootstrap.cc:492] T 10aa64ed664f40629f5f134052df8f5f P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
W20260924 00:21:13.338946 10359 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.126:34479, user_credentials={real_user=slave}} blocked reactor thread for 48867us
I20260924 00:21:13.339509 10773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73018242c4f842e59160eb188ede78dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:13.316874 10373 rpcz_store.cc:276] 0924 00:21:12.062891 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:12.167903 (+105012us) service_pool.cc:227] Handling call
0924 00:21:12.321719 (+153816us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b54000118d4. stack: 00007fde9ac98033 00007fde9ac97f30 00007fde9a819534 00000000004db2b4 00000000004db249 00007fdea1ba4af0 00007fdea1c27f2a 00007fdea3af3d6c 00007fdea3bef7d6 00007fdea1735634 00007fdea17355c2 00007fdea17354f1 00007fdea173546c 00007fdea1734722 00007fde9f9ce94c 00007fde9f9cdd86
0924 00:21:12.322349 (+ 630us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0924 00:21:12.323076 (+ 727us) catalog_manager.cc:3616] Apply alter schema
0924 00:21:12.323626 (+ 550us) catalog_manager.cc:3639] Acquired catalog manager lock
0924 00:21:12.323802 (+ 176us) catalog_manager.cc:3786] Updating metadata on disk
0924 00:21:12.415877 (+ 92075us) catalog_manager.cc:3817] Committing alterations to in-memory state
0924 00:21:13.316535 (+900658us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 00:21:12.324897 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 00:21:12.325249 (+ 352us) write_op.cc:432] Acquiring schema lock in shared mode
0924 00:21:12.325270 (+ 21us) write_op.cc:435] Acquired schema lock
0924 00:21:12.325282 (+ 12us) tablet.cc:660] Decoding operations
0924 00:21:12.325516 (+ 234us) write_op.cc:620] Acquiring the partition lock for write op
0924 00:21:12.325553 (+ 37us) write_op.cc:641] Partition lock acquired for write op
0924 00:21:12.325574 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0924 00:21:12.325703 (+ 129us) tablet.cc:699] Row locks acquired
0924 00:21:12.325716 (+ 13us) write_op.cc:260] PREPARE: finished
0924 00:21:12.325813 (+ 97us) write_op.cc:270] Start()
0924 00:21:12.325885 (+ 72us) write_op.cc:276] Timestamp: P: 1790209272325735 usec, L: 0
0924 00:21:12.325901 (+ 16us) op_driver.cc:331] REPLICATION: starting
0924 00:21:12.326351 (+ 450us) log.cc:842] Serialized 405 byte log entry
0924 00:21:12.398955 (+ 72604us) op_driver.cc:444] REPLICATION: finished
0924 00:21:12.399234 (+ 279us) write_op.cc:301] APPLY: starting
0924 00:21:12.399280 (+ 46us) tablet.cc:1365] starting BulkCheckPresence
0924 00:21:12.399365 (+ 85us) tablet.cc:1368] finished BulkCheckPresence
0924 00:21:12.399453 (+ 88us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 00:21:12.399594 (+ 141us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 00:21:12.399735 (+ 141us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0924 00:21:12.399774 (+ 39us) write_op.cc:312] APPLY: finished
0924 00:21:12.399931 (+ 157us) log.cc:842] Serialized 23 byte log entry
0924 00:21:12.400055 (+ 124us) write_op.cc:489] Releasing partition, row and schema locks
0924 00:21:12.400108 (+ 53us) write_op.cc:454] Released schema lock
0924 00:21:12.400138 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0924 00:21:12.400286 (+ 148us) write_op.cc:489] Releasing partition, row and schema locks
0924 00:21:12.400308 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":331411328,"child_traces":[["op",{"apply.queue_time_us":146,"apply.run_cpu_time_us":1204,"apply.run_wall_time_us":1203,"num_ops":1,"prepare.queue_time_us":86,"prepare.run_cpu_time_us":1715,"prepare.run_wall_time_us":1716,"raft.queue_time_us":238,"raft.run_cpu_time_us":584,"raft.run_wall_time_us":583,"replication_time_us":72988}]]}
I20260924 00:21:13.346402 10634 tablet_service.cc:1467] Processing CreateTablet for tablet 1cc672cf88554772bc4470d6120db6f6 (DEFAULT_TABLE table=default.Table_f1ae1e8ac652481680491d0da3355484 [id=36d61d1504cf495a9643bad809812d43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:13.346638 10910 tablet_service.cc:1514] Processing DeleteTablet for tablet 59a0a950a8144e99b2ed47be9a567d9a with delete_type TABLET_DATA_DELETED (replaced by tablet d4e120acff3e4c30aed813b0b24fbfee) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:13.346421 10625 tablet_service.cc:1467] Processing CreateTablet for tablet 73018242c4f842e59160eb188ede78dc (DEFAULT_TABLE table=default.Table_acfad28894184501be9e3f9e38fea952 [id=cdcf0facca9849d990e8e9463f288410]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:13.347622 10910 tablet_service.cc:1514] Processing DeleteTablet for tablet e937cee22326474598cef5c1d09b59c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:07 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:13.348229 11717 tablet_replica.cc:330] T e937cee22326474598cef5c1d09b59c2 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:13.348475 10910 tablet_service.cc:1514] Processing DeleteTablet for tablet b36cbb6520de4df58b88cfa53c20ee7a with delete_type TABLET_DATA_DELETED (replaced by tablet 9e44244181324288ad04afb7096839cb) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:13.349043 11717 raft_consensus.cc:2229] T e937cee22326474598cef5c1d09b59c2 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:21:13.349751 11717 pending_rounds.cc:62] T e937cee22326474598cef5c1d09b59c2 P c8626e300d0644b0908797ad61bd7cef: Trying to abort 1 pending ops.
I20260924 00:21:13.349799 10910 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b8d74761b0b4fd0a22021d3870ec05c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:07 UTC) from {username='slave'} at 127.0.0.1:35358
W20260924 00:21:13.350023 10358 catalog_manager.cc:4808] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 59a0a950a8144e99b2ed47be9a567d9a because the tablet was not found. No further retry: Not found: Tablet not found: 59a0a950a8144e99b2ed47be9a567d9a
I20260924 00:21:13.350009 11717 pending_rounds.cc:68] T e937cee22326474598cef5c1d09b59c2 P c8626e300d0644b0908797ad61bd7cef: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332697180425736192 op_type: NO_OP noop_request { }
I20260924 00:21:13.350405 11717 raft_consensus.cc:2866] T e937cee22326474598cef5c1d09b59c2 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260924 00:21:13.350723 11717 raft_consensus.cc:2258] T e937cee22326474598cef5c1d09b59c2 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:13.352957 11717 ts_tablet_manager.cc:1905] T e937cee22326474598cef5c1d09b59c2 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260924 00:21:13.358618 10361 catalog_manager.cc:4808] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet a72742864a6b4299b71757fb308dcadf because the tablet was not found. No further retry: Not found: Tablet not found: a72742864a6b4299b71757fb308dcadf
I20260924 00:21:13.359558 11716 tablet_replica.cc:330] T a0f71049f6b3478da3b07ebc9293ce64 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:13.360416 11716 raft_consensus.cc:2229] T a0f71049f6b3478da3b07ebc9293ce64 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:13.360930 11716 raft_consensus.cc:2258] T a0f71049f6b3478da3b07ebc9293ce64 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260924 00:21:13.361512 10361 catalog_manager.cc:4808] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 60ebab3e93f54383b87aeb54bbde722d because the tablet was not found. No further retry: Not found: Tablet not found: 60ebab3e93f54383b87aeb54bbde722d
I20260924 00:21:13.377349 11731 raft_consensus.cc:482] T 5b43ce1fed0749b8b076b9b026914e07 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:13.377324 10780 tablet_service.cc:1467] Processing CreateTablet for tablet bad371e91cb8460389a72e7beff7816d (DEFAULT_TABLE table=default.Table_8ff1c7565eff465da9ae77704e862894 [id=94267d66c5a74dd38e9f005f77b1ee18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:13.377779 11731 raft_consensus.cc:504] T 5b43ce1fed0749b8b076b9b026914e07 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:13.378309 10780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bad371e91cb8460389a72e7beff7816d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:13.379558 11731 leader_election.cc:288] T 5b43ce1fed0749b8b076b9b026914e07 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:21:13.381070 10920 tablet_service.cc:1514] Processing DeleteTablet for tablet cb51d5c6460049108cd7e1974821e578 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:06 UTC) from {username='slave'} at 127.0.0.1:35358
W20260924 00:21:13.385010 10374 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.4.67:49569 (request call id 66) took 1250 ms (client timeout 1000 ms). Trace:
W20260924 00:21:13.385430 10374 rpcz_store.cc:269] 0924 00:21:12.095577 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:12.157799 (+ 62222us) service_pool.cc:227] Handling call
0924 00:21:12.325369 (+167570us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b54000118d4. stack: 00007fde9ac98033 00007fde9ac97f30 00007fde9a819534 00000000004db2b4 00000000004db249 00007fdea1ba4af0 00007fdea1c27f2a 00007fdea3af3d6c 00007fdea3bedc3e 00007fdea1724fb4 00007fdea1724f42 00007fdea1724e71 00007fdea1724dec 00007fdea17240a2 00007fde9f9ce94c 00007fde9f9cdd86
0924 00:21:13.345696 (+1020327us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 00:21:13.159924 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 00:21:13.160243 (+ 319us) write_op.cc:432] Acquiring schema lock in shared mode
0924 00:21:13.160274 (+ 31us) write_op.cc:435] Acquired schema lock
0924 00:21:13.160286 (+ 12us) tablet.cc:660] Decoding operations
0924 00:21:13.160524 (+ 238us) write_op.cc:620] Acquiring the partition lock for write op
0924 00:21:13.160559 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0924 00:21:13.160580 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0924 00:21:13.160707 (+ 127us) tablet.cc:699] Row locks acquired
0924 00:21:13.160719 (+ 12us) write_op.cc:260] PREPARE: finished
0924 00:21:13.160816 (+ 97us) write_op.cc:270] Start()
0924 00:21:13.160891 (+ 75us) write_op.cc:276] Timestamp: P: 1790209273160757 usec, L: 0
0924 00:21:13.160905 (+ 14us) op_driver.cc:331] REPLICATION: starting
0924 00:21:13.161362 (+ 457us) log.cc:842] Serialized 527 byte log entry
0924 00:21:13.206603 (+ 45241us) op_driver.cc:444] REPLICATION: finished
0924 00:21:13.209503 (+ 2900us) write_op.cc:301] APPLY: starting
0924 00:21:13.209551 (+ 48us) tablet.cc:1365] starting BulkCheckPresence
0924 00:21:13.209631 (+ 80us) tablet.cc:1368] finished BulkCheckPresence
0924 00:21:13.209706 (+ 75us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 00:21:13.209835 (+ 129us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 00:21:13.209944 (+ 109us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0924 00:21:13.229534 (+ 19590us) write_op.cc:312] APPLY: finished
0924 00:21:13.229701 (+ 167us) log.cc:842] Serialized 23 byte log entry
0924 00:21:13.229828 (+ 127us) write_op.cc:489] Releasing partition, row and schema locks
0924 00:21:13.229888 (+ 60us) write_op.cc:454] Released schema lock
0924 00:21:13.229916 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0924 00:21:13.241541 (+ 11625us) write_op.cc:489] Releasing partition, row and schema locks
0924 00:21:13.241573 (+ 32us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":12,"spinlock_wait_cycles":379673856,"child_traces":[["op",{"apply.queue_time_us":2787,"apply.run_cpu_time_us":1273,"apply.run_wall_time_us":32220,"num_ops":1,"prepare.queue_time_us":56826,"prepare.run_cpu_time_us":1691,"prepare.run_wall_time_us":85230,"replication_time_us":45635,"thread_start_us":8895,"threads_started":1}]]}
I20260924 00:21:13.390681 11040 tablet_bootstrap.cc:654] T 10aa64ed664f40629f5f134052df8f5f P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:13.393482 11716 ts_tablet_manager.cc:1905] T a0f71049f6b3478da3b07ebc9293ce64 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:13.399384 10633 tablet_service.cc:1467] Processing CreateTablet for tablet bad371e91cb8460389a72e7beff7816d (DEFAULT_TABLE table=default.Table_8ff1c7565eff465da9ae77704e862894 [id=94267d66c5a74dd38e9f005f77b1ee18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:13.402314 10799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b43ce1fed0749b8b076b9b026914e07" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:13.416705 10656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b43ce1fed0749b8b076b9b026914e07" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:13.417322 10656 raft_consensus.cc:2454] T 5b43ce1fed0749b8b076b9b026914e07 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
I20260924 00:21:13.420442 10876 leader_election.cc:302] T 5b43ce1fed0749b8b076b9b026914e07 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:13.421797 10634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1cc672cf88554772bc4470d6120db6f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:13.425992 10369 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_dc51cf031b5e41ccb485d5db42ec823a" } modify_external_catalogs: true
I20260924 00:21:13.427186 10369 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_dc51cf031b5e41ccb485d5db42ec823a" } modify_external_catalogs: true
I20260924 00:21:13.430675 11041 tablet_bootstrap.cc:492] T cd30f52583124543b5d616ef94b31831 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:13.434931 11718 raft_consensus.cc:482] T 57ae3d10846a4aaa84eba81b13aafd44 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:13.435333 11718 raft_consensus.cc:504] T 57ae3d10846a4aaa84eba81b13aafd44 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:13.438207 11718 leader_election.cc:288] T 57ae3d10846a4aaa84eba81b13aafd44 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:13.449177 11041 tablet_bootstrap.cc:654] T cd30f52583124543b5d616ef94b31831 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:13.450552 10774 tablet_service.cc:1467] Processing CreateTablet for tablet fade2106cef748c5adc2af770027df63 (DEFAULT_TABLE table=default.Table_e2da51f4110147169f1bc08241696fae [id=c02abe55b70f4e23bc4a9f21300f39d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:13.451038 10627 tablet_service.cc:1467] Processing CreateTablet for tablet fade2106cef748c5adc2af770027df63 (DEFAULT_TABLE table=default.Table_e2da51f4110147169f1bc08241696fae [id=c02abe55b70f4e23bc4a9f21300f39d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:13.451467 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fade2106cef748c5adc2af770027df63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:13.452039 10627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fade2106cef748c5adc2af770027df63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:13.455569 10366 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_b89f627e972b4c3e841bdb133667aa66"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:13.456965 10366 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b89f627e972b4c3e841bdb133667aa66 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:13.463408 10368 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_d56dcf7ebba04b769e2e233cc7d9fa0e" } modify_external_catalogs: true
I20260924 00:21:13.462055 10799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57ae3d10846a4aaa84eba81b13aafd44" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:13.464028 10368 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_d56dcf7ebba04b769e2e233cc7d9fa0e" } modify_external_catalogs: true
W20260924 00:21:13.465732 10592 leader_election.cc:341] T 57ae3d10846a4aaa84eba81b13aafd44 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:13.468277 11797 consensus_queue.cc:1035] T 6bcfd729e9bd4040b3384ab24164e80d P c8626e300d0644b0908797ad61bd7cef [LEADER]: Connected to new peer: Peer: permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:13.469453 11797 raft_consensus.cc:2781] T 5b43ce1fed0749b8b076b9b026914e07 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:13.469725 11797 raft_consensus.cc:482] T 5b43ce1fed0749b8b076b9b026914e07 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:13.469781 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57ae3d10846a4aaa84eba81b13aafd44" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:13.470033 11797 raft_consensus.cc:3037] T 5b43ce1fed0749b8b076b9b026914e07 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:13.473008 10625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73018242c4f842e59160eb188ede78dc. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:21:13.481928 10589 leader_election.cc:341] T 57ae3d10846a4aaa84eba81b13aafd44 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:13.482318 10589 leader_election.cc:302] T 57ae3d10846a4aaa84eba81b13aafd44 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:13.483315 11718 raft_consensus.cc:2726] T 57ae3d10846a4aaa84eba81b13aafd44 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:13.483374 10367 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet 72d3bb80e1034881b94eb75de5d900c2 (table default.Table_b7b4d0037f4946659915a6bd5d65b202 [id=ca942906de0f4019bbdf72ec886fe6fb]). Expected version 3 got 0
I20260924 00:21:13.484154 10367 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet 4e9f00abffe14046bf47a6cae7010690 (table default.Table_79486769678c44fe98623f5aa06975bb [id=423a3ef82b0f488fa8fb082d4e6f63a3]). Expected version 3 got 0
I20260924 00:21:13.487959 10633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bad371e91cb8460389a72e7beff7816d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:13.492892 10773 tablet_service.cc:1467] Processing CreateTablet for tablet 6b8482823b4c40f0aedbf19d388d8013 (DEFAULT_TABLE table=default.Table_9e32aedf83114328ab54bc931f2a51a3 [id=e2acb1d9099a48798111f605b72f4d53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:13.493775 11771 raft_consensus.cc:482] T 59a0a950a8144e99b2ed47be9a567d9a P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:13.494177 11771 raft_consensus.cc:504] T 59a0a950a8144e99b2ed47be9a567d9a P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:13.495947 11771 leader_election.cc:288] T 59a0a950a8144e99b2ed47be9a567d9a P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:13.499114 11040 tablet_bootstrap.cc:492] T 10aa64ed664f40629f5f134052df8f5f P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:13.499518 11040 ts_tablet_manager.cc:1397] T 10aa64ed664f40629f5f134052df8f5f P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.164s user 0.007s sys 0.000s
I20260924 00:21:13.501667 11040 raft_consensus.cc:348] T 10aa64ed664f40629f5f134052df8f5f P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:13.502300 11040 raft_consensus.cc:374] T 10aa64ed664f40629f5f134052df8f5f P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:13.502516 11040 raft_consensus.cc:729] T 10aa64ed664f40629f5f134052df8f5f P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:13.503015 11040 consensus_queue.cc:260] T 10aa64ed664f40629f5f134052df8f5f P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:13.499122 10777 tablet_service.cc:1467] Processing CreateTablet for tablet c889f258d75448faafa7bb65e2aafc12 (DEFAULT_TABLE table=default.Table_e06918102eb24c628d528f20522dd2a4 [id=0b40ac1ec523496badcb50a185abeaf6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:13.484722 10367 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet 6bcfd729e9bd4040b3384ab24164e80d (table default.Table_9670783a92074b639ecb0d6c374811e5 [id=df2a5a74db65431c9ecb750184f64709]). Expected version 1 got 0
I20260924 00:21:13.509784 10773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b8482823b4c40f0aedbf19d388d8013. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:13.513017 11806 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b5aa3debe1cd45e19a6bbd0070913ac3)
I20260924 00:21:13.513649 11806 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } }
I20260924 00:21:13.516428 10309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" candidate_term: 3 candidate_status { last_received { term: 2 index: 322 } } ignore_live_leader: false dest_uuid: "1d68651913494ce8afee3c136b2da9cc" is_pre_election: true
I20260924 00:21:13.519078 11806 leader_election.cc:288] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1d68651913494ce8afee3c136b2da9cc (127.10.4.126:34479), b5aa3debe1cd45e19a6bbd0070913ac3 (127.10.4.125:34487)
I20260924 00:21:13.519268 10384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" candidate_term: 3 candidate_status { last_received { term: 2 index: 322 } } ignore_live_leader: false dest_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" is_pre_election: true
I20260924 00:21:13.525592 11412 leader_election.cc:302] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f8bd0b3184f84f5f8da10c9c21472f92; no voters: 1d68651913494ce8afee3c136b2da9cc, b5aa3debe1cd45e19a6bbd0070913ac3
I20260924 00:21:13.527769 11806 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 00:21:13.475888 11039 consensus_queue.cc:260] T a62b85ebb2b84ae19cad3e3e2dd07a59 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:13.529758 10777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c889f258d75448faafa7bb65e2aafc12. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:21:13.530342 10732 leader_election.cc:341] T 59a0a950a8144e99b2ed47be9a567d9a P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Not found: Tablet not found: 59a0a950a8144e99b2ed47be9a567d9a
I20260924 00:21:13.530386 10371 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_5db49282d679494c84606725e942b198" } new_table_name: "default.Table_24047fc04a434959928e6148f187a672" modify_external_catalogs: true
I20260924 00:21:13.529299 11724 raft_consensus.cc:482] T a62b85ebb2b84ae19cad3e3e2dd07a59 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:13.504860 11040 ts_tablet_manager.cc:1428] T 10aa64ed664f40629f5f134052df8f5f P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260924 00:21:13.545188 11040 tablet_bootstrap.cc:492] T e937cee22326474598cef5c1d09b59c2 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:13.551584 10365 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 72d3bb80e1034881b94eb75de5d900c2 (table default.Table_b7b4d0037f4946659915a6bd5d65b202 [id=ca942906de0f4019bbdf72ec886fe6fb]). Expected version 3 got 0
I20260924 00:21:13.552268 10365 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 4e9f00abffe14046bf47a6cae7010690 (table default.Table_79486769678c44fe98623f5aa06975bb [id=423a3ef82b0f488fa8fb082d4e6f63a3]). Expected version 3 got 0
I20260924 00:21:13.552930 10365 catalog_manager.cc:5495] T 3c850d0474414d2486cd7bbd6568e936 P b10a9082a37b46f09d581944914c1670 reported cstate change: term changed from 0 to 1, leader changed from <none> to b10a9082a37b46f09d581944914c1670 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "b10a9082a37b46f09d581944914c1670" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } health_report { overall_health: HEALTHY } } }
I20260924 00:21:13.553936 11040 tablet_bootstrap.cc:654] T e937cee22326474598cef5c1d09b59c2 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:13.574138 11040 tablet_bootstrap.cc:492] T e937cee22326474598cef5c1d09b59c2 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:13.574523 11040 ts_tablet_manager.cc:1397] T e937cee22326474598cef5c1d09b59c2 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.029s user 0.011s sys 0.006s
I20260924 00:21:13.576522 11040 raft_consensus.cc:348] T e937cee22326474598cef5c1d09b59c2 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:13.576547 11797 raft_consensus.cc:504] T 5b43ce1fed0749b8b076b9b026914e07 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:13.577224 11040 raft_consensus.cc:374] T e937cee22326474598cef5c1d09b59c2 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:13.577469 11040 raft_consensus.cc:729] T e937cee22326474598cef5c1d09b59c2 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:13.578126 11040 consensus_queue.cc:260] T e937cee22326474598cef5c1d09b59c2 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:13.578886 11797 leader_election.cc:288] T 5b43ce1fed0749b8b076b9b026914e07 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:21:13.545563 11724 raft_consensus.cc:504] T a62b85ebb2b84ae19cad3e3e2dd07a59 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:13.580300 11040 ts_tablet_manager.cc:1428] T e937cee22326474598cef5c1d09b59c2 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260924 00:21:13.583395 11724 leader_election.cc:288] T a62b85ebb2b84ae19cad3e3e2dd07a59 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:13.587307 11040 tablet_bootstrap.cc:492] T 675b9e33edcf4d4ea7fa428b476e3945 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:13.597369 11040 tablet_bootstrap.cc:654] T 675b9e33edcf4d4ea7fa428b476e3945 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
W20260924 00:21:13.594666 10878 leader_election.cc:341] T 5b43ce1fed0749b8b076b9b026914e07 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:13.617980 10374 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_53040e3d85054d8b94a32de64fbedd7e" } new_table_name: "default.Table_045585fb6067443fbd323a3bbc688220" modify_external_catalogs: true
I20260924 00:21:13.623234 10369 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 8e628abffd9545b391c25f83c9903732
I20260924 00:21:13.623111 10794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b43ce1fed0749b8b076b9b026914e07" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:21:13.623226 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a62b85ebb2b84ae19cad3e3e2dd07a59" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:13.624526 11716 ts_tablet_manager.cc:1918] T a0f71049f6b3478da3b07ebc9293ce64 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:13.628116 10636 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e628abffd9545b391c25f83c9903732 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:13 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:13.629369 10799 raft_consensus.cc:3037] T 10aa64ed664f40629f5f134052df8f5f P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:13.633880 10772 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e628abffd9545b391c25f83c9903732 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:13 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:13.641108 10368 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 10aa64ed664f40629f5f134052df8f5f
W20260924 00:21:13.643008 10371 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet df03f899e5de4c12b16fbb40bdb1f33a consensus state has no leader
I20260924 00:21:13.624863 11716 log.cc:1196] T a0f71049f6b3478da3b07ebc9293ce64 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/a0f71049f6b3478da3b07ebc9293ce64
I20260924 00:21:13.651798 11716 ts_tablet_manager.cc:1939] T a0f71049f6b3478da3b07ebc9293ce64 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:13.653891 10772 tablet_service.cc:1514] Processing DeleteTablet for tablet 10aa64ed664f40629f5f134052df8f5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:13 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:13.661003 10368 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_d775e9e0f1614f9fbbae3dfd8554f4c2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:13.662670 10368 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d775e9e0f1614f9fbbae3dfd8554f4c2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:13.663637 10366 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_8f274639bca846bf89f8770c7de95055"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:13.668283 11039 ts_tablet_manager.cc:1428] T a62b85ebb2b84ae19cad3e3e2dd07a59 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.430s user 0.005s sys 0.002s
W20260924 00:21:13.668696 10366 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8f274639bca846bf89f8770c7de95055 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:13.669080 11039 tablet_bootstrap.cc:492] T cd30f52583124543b5d616ef94b31831 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:13.664841 10371 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_d1738e7eeecb4601a9402b1745b33bd3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:13.673532 10371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d1738e7eeecb4601a9402b1745b33bd3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:13.674130 10912 tablet_service.cc:1467] Processing CreateTablet for tablet bad371e91cb8460389a72e7beff7816d (DEFAULT_TABLE table=default.Table_8ff1c7565eff465da9ae77704e862894 [id=94267d66c5a74dd38e9f005f77b1ee18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:13.675120 10912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bad371e91cb8460389a72e7beff7816d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:13.675987 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet a0f71049f6b3478da3b07ebc9293ce64 (table default.Table_9f2df24361da4f9a9340f6f8673043ea [id=59ec3e94e2f44150994a5ffc1d9fa8f4]) successfully deleted
I20260924 00:21:13.676304 11716 tablet_replica.cc:330] T 0c583d9ba1414f8ea2e605b4b6b076f8 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:13.677114 11716 raft_consensus.cc:2229] T 0c583d9ba1414f8ea2e605b4b6b076f8 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:13.686982 11716 raft_consensus.cc:2258] T 0c583d9ba1414f8ea2e605b4b6b076f8 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:13.692858 11716 ts_tablet_manager.cc:1905] T 0c583d9ba1414f8ea2e605b4b6b076f8 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:13.693157 10908 tablet_service.cc:1467] Processing CreateTablet for tablet fade2106cef748c5adc2af770027df63 (DEFAULT_TABLE table=default.Table_e2da51f4110147169f1bc08241696fae [id=c02abe55b70f4e23bc4a9f21300f39d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:13.694370 10908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fade2106cef748c5adc2af770027df63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:13.704258 11710 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92: acquiring CA information for follower catalog manager: success
I20260924 00:21:13.705943 10906 tablet_service.cc:1467] Processing CreateTablet for tablet 6b8482823b4c40f0aedbf19d388d8013 (DEFAULT_TABLE table=default.Table_9e32aedf83114328ab54bc931f2a51a3 [id=e2acb1d9099a48798111f605b72f4d53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:13.706929 10906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b8482823b4c40f0aedbf19d388d8013. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:13.699355 11771 raft_consensus.cc:482] T 10aa64ed664f40629f5f134052df8f5f P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:13.709331 11771 raft_consensus.cc:504] T 10aa64ed664f40629f5f134052df8f5f P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:13.710610 10367 catalog_manager.cc:6040] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: default.Table_382099cf77a24bb8bd6fe084926431c9 [id=3db896d304514ead8dfe726c402a6fcd] alter complete (version 1)
I20260924 00:21:13.711210 11771 leader_election.cc:288] T 10aa64ed664f40629f5f134052df8f5f P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:13.713721 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10aa64ed664f40629f5f134052df8f5f" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:13.728015 10915 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e628abffd9545b391c25f83c9903732 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:13 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:13.729427 11717 ts_tablet_manager.cc:1918] T e937cee22326474598cef5c1d09b59c2 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:13.729791 11717 log.cc:1196] T e937cee22326474598cef5c1d09b59c2 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/e937cee22326474598cef5c1d09b59c2
I20260924 00:21:13.730511 11717 ts_tablet_manager.cc:1939] T e937cee22326474598cef5c1d09b59c2 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:13.733647 11039 tablet_bootstrap.cc:654] T cd30f52583124543b5d616ef94b31831 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:13.733961 10915 tablet_service.cc:1514] Processing DeleteTablet for tablet 10aa64ed664f40629f5f134052df8f5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:13 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:13.739689 10915 tablet_service.cc:1514] Processing DeleteTablet for tablet 675b9e33edcf4d4ea7fa428b476e3945 with delete_type TABLET_DATA_DELETED (replaced by tablet 915e8072851c4d2b90f8af12b1dc58e0) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:13.740824 10369 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet 6bcfd729e9bd4040b3384ab24164e80d (table default.Table_9670783a92074b639ecb0d6c374811e5 [id=df2a5a74db65431c9ecb750184f64709]). Expected version 1 got 0
I20260924 00:21:13.745693 10915 tablet_service.cc:1514] Processing DeleteTablet for tablet cf36d53828eb421d87b52b3e4cb1d748 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:05 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:13.747141 10915 tablet_service.cc:1467] Processing CreateTablet for tablet c889f258d75448faafa7bb65e2aafc12 (DEFAULT_TABLE table=default.Table_e06918102eb24c628d528f20522dd2a4 [id=0b40ac1ec523496badcb50a185abeaf6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:13.748121 10915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c889f258d75448faafa7bb65e2aafc12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:13.752386 11040 tablet_bootstrap.cc:492] T 675b9e33edcf4d4ea7fa428b476e3945 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:13.752812 11040 ts_tablet_manager.cc:1397] T 675b9e33edcf4d4ea7fa428b476e3945 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.166s user 0.013s sys 0.013s
I20260924 00:21:13.754629 11717 tablet_replica.cc:330] T 3b8d74761b0b4fd0a22021d3870ec05c P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:13.755019 11040 raft_consensus.cc:348] T 675b9e33edcf4d4ea7fa428b476e3945 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:13.755606 11040 raft_consensus.cc:374] T 675b9e33edcf4d4ea7fa428b476e3945 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:13.755836 11040 raft_consensus.cc:729] T 675b9e33edcf4d4ea7fa428b476e3945 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:13.756325 11040 consensus_queue.cc:260] T 675b9e33edcf4d4ea7fa428b476e3945 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
W20260924 00:21:13.762490 10374 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 667e84b0e2d1486289710ea36c594a66 consensus state has no leader
I20260924 00:21:13.768795 11771 raft_consensus.cc:482] T e937cee22326474598cef5c1d09b59c2 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:13.769229 11771 raft_consensus.cc:504] T e937cee22326474598cef5c1d09b59c2 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:13.769867 11040 ts_tablet_manager.cc:1428] T 675b9e33edcf4d4ea7fa428b476e3945 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.017s user 0.005s sys 0.000s
I20260924 00:21:13.769946 11717 raft_consensus.cc:2229] T 3b8d74761b0b4fd0a22021d3870ec05c P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:21:13.770900 11717 raft_consensus.cc:2258] T 3b8d74761b0b4fd0a22021d3870ec05c P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 00:21:13.773178 10732 leader_election.cc:341] T e937cee22326474598cef5c1d09b59c2 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Not found: Tablet not found: e937cee22326474598cef5c1d09b59c2
I20260924 00:21:13.774493 11771 leader_election.cc:288] T e937cee22326474598cef5c1d09b59c2 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:13.781025 10629 tablet_service.cc:1514] Processing DeleteTablet for tablet 717759eb61674c89b05b8d2b0dc34979 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:03 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:13.786553 11040 tablet_bootstrap.cc:492] T 3c850d0474414d2486cd7bbd6568e936 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:13.791759 11040 tablet_bootstrap.cc:654] T 3c850d0474414d2486cd7bbd6568e936 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:13.793717 10636 tablet_service.cc:1514] Processing DeleteTablet for tablet 10aa64ed664f40629f5f134052df8f5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:13 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:13.797375 10630 tablet_service.cc:1514] Processing DeleteTablet for tablet cf36d53828eb421d87b52b3e4cb1d748 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:05 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:13.797832 11777 raft_consensus.cc:482] T 717759eb61674c89b05b8d2b0dc34979 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 00:21:13.823167 10876 leader_election.cc:341] T a62b85ebb2b84ae19cad3e3e2dd07a59 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: a62b85ebb2b84ae19cad3e3e2dd07a59
I20260924 00:21:13.832088 10371 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_284132ac952a4806a25df1a18c9ab709" } modify_external_catalogs: true
I20260924 00:21:13.832656 10371 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_284132ac952a4806a25df1a18c9ab709" } modify_external_catalogs: true
I20260924 00:21:13.834599 11041 tablet_bootstrap.cc:492] T cd30f52583124543b5d616ef94b31831 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:13.835829 11041 ts_tablet_manager.cc:1397] T cd30f52583124543b5d616ef94b31831 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.405s user 0.023s sys 0.007s
I20260924 00:21:13.838507 10912 tablet_service.cc:1467] Processing CreateTablet for tablet fcccd9816d0642b5b4a9fc2d4be59464 (DEFAULT_TABLE table=default.Table_f9897a81fc084a25ae405522cf5c7ef9 [id=7a6f8b0af8484d8b80c02e80027db9d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:13.839145 10920 tablet_service.cc:1467] Processing CreateTablet for tablet 6235db91988345f799160251990487d2 (DEFAULT_TABLE table=default.Table_5cf852dcccf3498f96d0e5e8364d1a84 [id=a15434697c7b4707a0effb20cbb36609]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:13.840143 10920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6235db91988345f799160251990487d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:13.841161 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet e937cee22326474598cef5c1d09b59c2 (table default.Table_d2a991e0c92e4c73b70bfbfb8bea5d58 [id=54d197b284b34dc0a5eea440b0b1e5de]) successfully deleted
W20260924 00:21:13.842023 10358 catalog_manager.cc:4808] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet b36cbb6520de4df58b88cfa53c20ee7a because the tablet was not found. No further retry: Not found: Tablet not found: b36cbb6520de4df58b88cfa53c20ee7a
I20260924 00:21:13.843742 10368 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47022:
name: "default.Table_71eefae0e0f5450683c2e81bfef89116"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:13.845171 10368 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_71eefae0e0f5450683c2e81bfef89116 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:13.845937 10912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcccd9816d0642b5b4a9fc2d4be59464. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:13.857014 10636 tablet_service.cc:1514] Processing DeleteTablet for tablet 57ae3d10846a4aaa84eba81b13aafd44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:09 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:13.857632 10366 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_07390b277c404c7c9daecd17fe3dccca" } modify_external_catalogs: true
I20260924 00:21:13.858130 10366 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_07390b277c404c7c9daecd17fe3dccca" } modify_external_catalogs: true
I20260924 00:21:13.858819 10630 tablet_service.cc:1514] Processing DeleteTablet for tablet 9319f9caadb04067a2e9700b5019ed37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:04 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:13.859064 10624 tablet_service.cc:1467] Processing CreateTablet for tablet c889f258d75448faafa7bb65e2aafc12 (DEFAULT_TABLE table=default.Table_e06918102eb24c628d528f20522dd2a4 [id=0b40ac1ec523496badcb50a185abeaf6]), partition=RANGE (key) PARTITION UNBOUNDED
W20260924 00:21:13.859620 11804 log.cc:924] Time spent T 6bcfd729e9bd4040b3384ab24164e80d P b10a9082a37b46f09d581944914c1670: Append to log took a long time: real 0.370s user 0.000s sys 0.000s
I20260924 00:21:13.794289 11776 raft_consensus.cc:482] T 98926dd3850d40938b67779d5fd9a1db P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:13.799296 11777 raft_consensus.cc:504] T 717759eb61674c89b05b8d2b0dc34979 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:13.861392 10374 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_e0eff65c2f3e4800b2345a743d5e1a8a" } new_table_name: "default.Table_06b78e500bd24799a8632eac4b4b108c" modify_external_catalogs: true
W20260924 00:21:13.862056 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 8e628abffd9545b391c25f83c9903732 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8e628abffd9545b391c25f83c9903732 already in progress: creating tablet
I20260924 00:21:13.799853 10626 tablet_service.cc:1514] Processing DeleteTablet for tablet 5107b763acf04df88553ba0af2757570 with delete_type TABLET_DATA_DELETED (replaced by tablet cf0f0f016d074b1fb78d9b427c67fe86) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:13.860466 10649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10aa64ed664f40629f5f134052df8f5f" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:13.862923 10649 raft_consensus.cc:2454] T 10aa64ed664f40629f5f134052df8f5f P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 1.
I20260924 00:21:13.863230 10630 tablet_service.cc:1467] Processing CreateTablet for tablet 6235db91988345f799160251990487d2 (DEFAULT_TABLE table=default.Table_5cf852dcccf3498f96d0e5e8364d1a84 [id=a15434697c7b4707a0effb20cbb36609]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:13.860590 11776 raft_consensus.cc:504] T 98926dd3850d40938b67779d5fd9a1db P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:13.825199 10651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b43ce1fed0749b8b076b9b026914e07" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:21:13.865731 10651 raft_consensus.cc:3037] T 5b43ce1fed0749b8b076b9b026914e07 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:13.866111 11776 leader_election.cc:288] T 98926dd3850d40938b67779d5fd9a1db P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:13.867219 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98926dd3850d40938b67779d5fd9a1db" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
W20260924 00:21:13.868227 10589 leader_election.cc:341] T 98926dd3850d40938b67779d5fd9a1db P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:13.874298 10636 tablet_service.cc:1467] Processing CreateTablet for tablet fcccd9816d0642b5b4a9fc2d4be59464 (DEFAULT_TABLE table=default.Table_f9897a81fc084a25ae405522cf5c7ef9 [id=7a6f8b0af8484d8b80c02e80027db9d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:13.875525 11777 leader_election.cc:288] T 717759eb61674c89b05b8d2b0dc34979 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:13.879639 11719 tablet_replica.cc:330] T 717759eb61674c89b05b8d2b0dc34979 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:13.879705 10627 tablet_service.cc:1467] Processing CreateTablet for tablet 6b8482823b4c40f0aedbf19d388d8013 (DEFAULT_TABLE table=default.Table_9e32aedf83114328ab54bc931f2a51a3 [id=e2acb1d9099a48798111f605b72f4d53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:13.880582 11719 raft_consensus.cc:2229] T 717759eb61674c89b05b8d2b0dc34979 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:13.880806 10627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b8482823b4c40f0aedbf19d388d8013. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:13.881047 11719 raft_consensus.cc:2258] T 717759eb61674c89b05b8d2b0dc34979 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:13.882753 11719 ts_tablet_manager.cc:1905] T 717759eb61674c89b05b8d2b0dc34979 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:13.885423 10624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c889f258d75448faafa7bb65e2aafc12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:13.887794 10733 leader_election.cc:302] T 10aa64ed664f40629f5f134052df8f5f P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670; no voters: c8626e300d0644b0908797ad61bd7cef
W20260924 00:21:13.888988 10733 leader_election.cc:341] T 59a0a950a8144e99b2ed47be9a567d9a P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: 59a0a950a8144e99b2ed47be9a567d9a
I20260924 00:21:13.889320 10733 leader_election.cc:302] T 59a0a950a8144e99b2ed47be9a567d9a P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08; no voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:13.890345 10630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6235db91988345f799160251990487d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:13.894773 11801 consensus_queue.cc:1035] T 10aa64ed664f40629f5f134052df8f5f P c8626e300d0644b0908797ad61bd7cef [LEADER]: Connected to new peer: Peer: permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:13.903398 10772 tablet_service.cc:1514] Processing DeleteTablet for tablet a0f71049f6b3478da3b07ebc9293ce64 with delete_type TABLET_DATA_DELETED (replaced by tablet d574642ebc6340fcb5e83c36804ee9f8) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:13.909251 10772 tablet_service.cc:1514] Processing DeleteTablet for tablet cb51d5c6460049108cd7e1974821e578 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:06 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:13.909938 11771 raft_consensus.cc:2781] T 10aa64ed664f40629f5f134052df8f5f P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 00:21:13.910372 11771 raft_consensus.cc:482] T 10aa64ed664f40629f5f134052df8f5f P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:13.910701 11771 raft_consensus.cc:3037] T 10aa64ed664f40629f5f134052df8f5f P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Advancing to term 2
I20260924 00:21:13.913858 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "717759eb61674c89b05b8d2b0dc34979" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:13.907773 11717 ts_tablet_manager.cc:1905] T 3b8d74761b0b4fd0a22021d3870ec05c P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260924 00:21:13.921146 10589 leader_election.cc:341] T 717759eb61674c89b05b8d2b0dc34979 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:13.923142 11041 raft_consensus.cc:348] T cd30f52583124543b5d616ef94b31831 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:13.923776 11041 raft_consensus.cc:374] T cd30f52583124543b5d616ef94b31831 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:13.924005 11041 raft_consensus.cc:729] T cd30f52583124543b5d616ef94b31831 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:13.924618 11041 consensus_queue.cc:260] T cd30f52583124543b5d616ef94b31831 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:13.904839 10651 raft_consensus.cc:2454] T 5b43ce1fed0749b8b076b9b026914e07 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 1.
I20260924 00:21:13.927229 10876 leader_election.cc:302] T 5b43ce1fed0749b8b076b9b026914e07 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:13.928086 11801 raft_consensus.cc:2781] T 5b43ce1fed0749b8b076b9b026914e07 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:13.928514 11801 raft_consensus.cc:686] T 5b43ce1fed0749b8b076b9b026914e07 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Becoming Leader. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Running, Role: LEADER
I20260924 00:21:13.928606 11822 raft_consensus.cc:2726] T 59a0a950a8144e99b2ed47be9a567d9a P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:13.929306 11801 consensus_queue.cc:237] T 5b43ce1fed0749b8b076b9b026914e07 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
W20260924 00:21:13.933147 10733 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:33461, user_credentials={real_user=slave}} blocked reactor thread for 44197.4us
W20260924 00:21:13.934126 10733 leader_election.cc:341] T e937cee22326474598cef5c1d09b59c2 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: e937cee22326474598cef5c1d09b59c2
I20260924 00:21:13.934566 10772 tablet_service.cc:1514] Processing DeleteTablet for tablet b36cbb6520de4df58b88cfa53c20ee7a with delete_type TABLET_DATA_DELETED (replaced by tablet 9e44244181324288ad04afb7096839cb) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:13.935097 10777 tablet_service.cc:1514] Processing DeleteTablet for tablet 59a0a950a8144e99b2ed47be9a567d9a with delete_type TABLET_DATA_DELETED (replaced by tablet d4e120acff3e4c30aed813b0b24fbfee) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:13.934556 10733 leader_election.cc:302] T e937cee22326474598cef5c1d09b59c2 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08; no voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:13.935372 10772 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b8d74761b0b4fd0a22021d3870ec05c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:07 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:13.936798 10772 tablet_service.cc:1467] Processing CreateTablet for tablet fcccd9816d0642b5b4a9fc2d4be59464 (DEFAULT_TABLE table=default.Table_f9897a81fc084a25ae405522cf5c7ef9 [id=7a6f8b0af8484d8b80c02e80027db9d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:13.937798 10772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcccd9816d0642b5b4a9fc2d4be59464. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:13.944530 10371 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 1492ece2346d461ea57096a897daf83f
I20260924 00:21:13.945995 10777 tablet_service.cc:1467] Processing CreateTablet for tablet 6235db91988345f799160251990487d2 (DEFAULT_TABLE table=default.Table_5cf852dcccf3498f96d0e5e8364d1a84 [id=a15434697c7b4707a0effb20cbb36609]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:13.946964 10777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6235db91988345f799160251990487d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:13.951713 10636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcccd9816d0642b5b4a9fc2d4be59464. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:13.954727 10780 tablet_service.cc:1514] Processing DeleteTablet for tablet e937cee22326474598cef5c1d09b59c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:07 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:13.957165 10773 tablet_service.cc:1514] Processing DeleteTablet for tablet 675b9e33edcf4d4ea7fa428b476e3945 with delete_type TABLET_DATA_DELETED (replaced by tablet 915e8072851c4d2b90f8af12b1dc58e0) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:13.958071 10773 tablet_service.cc:1514] Processing DeleteTablet for tablet 1492ece2346d461ea57096a897daf83f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:13 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:13.961294 11815 raft_consensus.cc:2726] T e937cee22326474598cef5c1d09b59c2 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:13.962080 11815 raft_consensus.cc:482] T 675b9e33edcf4d4ea7fa428b476e3945 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 00:21:13.962476 10358 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 667e84b0e2d1486289710ea36c594a66 (table default.Table_045585fb6067443fbd323a3bbc688220 [id=1c3a915542a1470785614e97b5779149], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 667e84b0e2d1486289710ea36c594a66 consensus state has no leader
I20260924 00:21:13.962811 11815 raft_consensus.cc:504] T 675b9e33edcf4d4ea7fa428b476e3945 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:13.968171 11771 raft_consensus.cc:504] T 10aa64ed664f40629f5f134052df8f5f P 48fbd603abe146c585d15196e1b5be08 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
W20260924 00:21:13.969889 10878 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:35359, user_credentials={real_user=slave}} blocked reactor thread for 130479us
W20260924 00:21:13.970543 10878 leader_election.cc:341] T 5b43ce1fed0749b8b076b9b026914e07 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
W20260924 00:21:13.972015 10878 leader_election.cc:341] T a62b85ebb2b84ae19cad3e3e2dd07a59 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:13.972038 10371 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47022:
table { table_name: "default.Table_e3fed8d230874b52bf545da40daf0c48" } new_table_name: "default.Table_97629e88aeda47a880934c90cc33aaf6" modify_external_catalogs: true
I20260924 00:21:13.972498 10878 leader_election.cc:302] T a62b85ebb2b84ae19cad3e3e2dd07a59 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670
I20260924 00:21:13.983754 10630 tablet_service.cc:1514] Processing DeleteTablet for tablet 1492ece2346d461ea57096a897daf83f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:13 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:13.996212 10624 tablet_service.cc:1467] Processing CreateTablet for tablet 36b1bc216f964b898503712eda33eeb6 (DEFAULT_TABLE table=default.Table_eff603db4cdf417c871be7eb94681227 [id=ccd80429571145e0bcb1ade6e3a315d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:13.999441 11814 raft_consensus.cc:2726] T a62b85ebb2b84ae19cad3e3e2dd07a59 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:13.999879 10257 master-stress-test.cc:578] Killing test thread 1/10
I20260924 00:21:14.000054 11814 consensus_queue.cc:1035] T 10aa64ed664f40629f5f134052df8f5f P c8626e300d0644b0908797ad61bd7cef [LEADER]: Connected to new peer: Peer: permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:14.005107 10624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36b1bc216f964b898503712eda33eeb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:14.006866 11718 raft_consensus.cc:482] T 4c2a729a802e4abe9ab4f989a40198be P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:14.007279 11718 raft_consensus.cc:504] T 4c2a729a802e4abe9ab4f989a40198be P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:14.020718 11039 tablet_bootstrap.cc:492] T cd30f52583124543b5d616ef94b31831 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:14.019671 11717 ts_tablet_manager.cc:1918] T 3b8d74761b0b4fd0a22021d3870ec05c P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:14.021397 11717 log.cc:1196] T 3b8d74761b0b4fd0a22021d3870ec05c P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/3b8d74761b0b4fd0a22021d3870ec05c
I20260924 00:21:14.021790 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98926dd3850d40938b67779d5fd9a1db" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:14.022269 11717 ts_tablet_manager.cc:1939] T 3b8d74761b0b4fd0a22021d3870ec05c P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:14.024325 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "717759eb61674c89b05b8d2b0dc34979" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:14.009032 11718 leader_election.cc:288] T 4c2a729a802e4abe9ab4f989a40198be P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:13.998384 11719 ts_tablet_manager.cc:1918] T 717759eb61674c89b05b8d2b0dc34979 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:14.026851 11719 log.cc:1196] T 717759eb61674c89b05b8d2b0dc34979 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/717759eb61674c89b05b8d2b0dc34979
I20260924 00:21:14.027521 11719 ts_tablet_manager.cc:1939] T 717759eb61674c89b05b8d2b0dc34979 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:14.028553 11719 tablet_replica.cc:330] T 10aa64ed664f40629f5f134052df8f5f P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:14.029214 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 717759eb61674c89b05b8d2b0dc34979 (table default.Table_9b991221b38645378eda8c1403ac0ce7 [id=b22dcf8c1c4346bebde8d0b9af8cf77c]) successfully deleted
W20260924 00:21:14.029969 10733 consensus_peers.cc:487] T cb51d5c6460049108cd7e1974821e578 P 48fbd603abe146c585d15196e1b5be08 -> Peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Couldn't send request to peer b10a9082a37b46f09d581944914c1670. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cb51d5c6460049108cd7e1974821e578. This is attempt 1: this message will repeat every 5th retry.
I20260924 00:21:14.024900 11717 tablet_replica.cc:330] T cb51d5c6460049108cd7e1974821e578 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:14.028759 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c2a729a802e4abe9ab4f989a40198be" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
W20260924 00:21:14.037290 10589 leader_election.cc:341] T 4c2a729a802e4abe9ab4f989a40198be P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:14.037706 11717 raft_consensus.cc:2229] T cb51d5c6460049108cd7e1974821e578 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:14.038163 11717 raft_consensus.cc:2258] T cb51d5c6460049108cd7e1974821e578 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:14.038232 11719 raft_consensus.cc:2229] T 10aa64ed664f40629f5f134052df8f5f P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:14.038854 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10aa64ed664f40629f5f134052df8f5f" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef"
I20260924 00:21:14.039883 11717 ts_tablet_manager.cc:1905] T cb51d5c6460049108cd7e1974821e578 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:14.040395 11719 raft_consensus.cc:2258] T 10aa64ed664f40629f5f134052df8f5f P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:14.042419 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "675b9e33edcf4d4ea7fa428b476e3945" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:14.043310 10942 raft_consensus.cc:2454] T 675b9e33edcf4d4ea7fa428b476e3945 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 0.
I20260924 00:21:14.044097 10732 leader_election.cc:302] T 675b9e33edcf4d4ea7fa428b476e3945 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:14.053337 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 3b8d74761b0b4fd0a22021d3870ec05c (table default.Table_0cc93ce7fcfb44089209422666978c62 [id=c0cd199f74494deda9d088345c4c90a3]) successfully deleted
I20260924 00:21:14.055898 11771 leader_election.cc:288] T 10aa64ed664f40629f5f134052df8f5f P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 2 election: Requested vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:14.057010 11041 ts_tablet_manager.cc:1428] T cd30f52583124543b5d616ef94b31831 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.221s user 0.004s sys 0.002s
I20260924 00:21:14.057816 11041 tablet_bootstrap.cc:492] T 2e839fd66fed4472a0892baa9aa85c54 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:14.061096 11719 ts_tablet_manager.cc:1905] T 10aa64ed664f40629f5f134052df8f5f P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:14.062357 10365 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 6bcfd729e9bd4040b3384ab24164e80d (table default.Table_9670783a92074b639ecb0d6c374811e5 [id=df2a5a74db65431c9ecb750184f64709]). Expected version 1 got 0
I20260924 00:21:14.062721 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10aa64ed664f40629f5f134052df8f5f" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:21:14.062999 10365 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 4c2a729a802e4abe9ab4f989a40198be (table default.Table_5816c202c13e439cbd04eb58f456ebb5 [id=136a32ddaf2a4d2fbdddecf3ea95970c]). Expected version 2 got 0
I20260924 00:21:14.021133 11039 ts_tablet_manager.cc:1397] T cd30f52583124543b5d616ef94b31831 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.352s user 0.017s sys 0.000s
I20260924 00:21:14.061615 11822 raft_consensus.cc:2781] T 675b9e33edcf4d4ea7fa428b476e3945 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260924 00:21:14.063975 10733 leader_election.cc:341] T 675b9e33edcf4d4ea7fa428b476e3945 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: 675b9e33edcf4d4ea7fa428b476e3945
W20260924 00:21:14.067092 10733 leader_election.cc:341] T 10aa64ed664f40629f5f134052df8f5f P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Illegal state: Tablet not RUNNING: STOPPED
I20260924 00:21:14.067451 10733 leader_election.cc:302] T 10aa64ed664f40629f5f134052df8f5f P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08; no voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:14.067727 10630 tablet_service.cc:1467] Processing CreateTablet for tablet 004b12724c404a6fbedfd03fe6da2b2b (DEFAULT_TABLE table=default.Table_e33891e1d1b745819f1114e54bded8bb [id=b382a8277aa3452990df0abe27a80cb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:14.068362 11822 raft_consensus.cc:482] T 675b9e33edcf4d4ea7fa428b476e3945 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:14.068614 10630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 004b12724c404a6fbedfd03fe6da2b2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:14.068768 11822 raft_consensus.cc:3037] T 675b9e33edcf4d4ea7fa428b476e3945 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:14.070111 10369 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 4e9f00abffe14046bf47a6cae7010690 (table default.Table_79486769678c44fe98623f5aa06975bb [id=423a3ef82b0f488fa8fb082d4e6f63a3]). Expected version 3 got 0
I20260924 00:21:14.070962 10369 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 6bcfd729e9bd4040b3384ab24164e80d (table default.Table_9670783a92074b639ecb0d6c374811e5 [id=df2a5a74db65431c9ecb750184f64709]). Expected version 1 got 0
I20260924 00:21:14.071465 10369 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 1d5a76bc420b41098d4ebd3dc59aa1fc (table default.Table_efa7000cc59a43639d964cd555a08961 [id=ff77e539ecb44767bb76cf076799a4c2]). Expected version 1 got 0
I20260924 00:21:14.064672 11815 leader_election.cc:288] T 675b9e33edcf4d4ea7fa428b476e3945 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:14.112131 10367 master_service.cc:946] ReplaceTablet: received request to replace tablet 3dae8b8d7743416983e9e903ea57140d from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:14.129678 10906 tablet_service.cc:1514] Processing DeleteTablet for tablet 1492ece2346d461ea57096a897daf83f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:13 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:14.130972 10906 tablet_service.cc:1467] Processing CreateTablet for tablet 36b1bc216f964b898503712eda33eeb6 (DEFAULT_TABLE table=default.Table_eff603db4cdf417c871be7eb94681227 [id=ccd80429571145e0bcb1ade6e3a315d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:14.131877 10906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36b1bc216f964b898503712eda33eeb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:14.136660 10366 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 85a0ad61b7cb4e11b30e84ddc4655b64
I20260924 00:21:14.137859 10365 catalog_manager.cc:5587] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461) does not have the latest schema for tablet 6bcfd729e9bd4040b3384ab24164e80d (table default.Table_9670783a92074b639ecb0d6c374811e5 [id=df2a5a74db65431c9ecb750184f64709]). Expected version 1 got 0
W20260924 00:21:14.138662 10374 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f220c1fadaf046f6bf085fcdf87109dd has no consensus state
W20260924 00:21:14.143853 10371 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ba31383ba3a04edab4c250c4b3146185 consensus state has no leader
I20260924 00:21:14.147816 11834 tablet.cc:1722] T 72d3bb80e1034881b94eb75de5d900c2 P c8626e300d0644b0908797ad61bd7cef: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260924 00:21:14.156694 11832 tablet.cc:1722] T 4e9f00abffe14046bf47a6cae7010690 P b10a9082a37b46f09d581944914c1670: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260924 00:21:14.161486 10367 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 3dae8b8d7743416983e9e903ea57140d
I20260924 00:21:14.160632 11822 raft_consensus.cc:504] T 675b9e33edcf4d4ea7fa428b476e3945 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:14.162744 10367 catalog_manager.cc:6565] ReplaceTablet: tablet 3dae8b8d7743416983e9e903ea57140d deleted and replaced by tablet 08929edc5c5b4ecd80cef4fc472a9ffa
I20260924 00:21:14.162638 11776 raft_consensus.cc:482] T cd30f52583124543b5d616ef94b31831 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:14.162825 10633 tablet_service.cc:1514] Processing DeleteTablet for tablet cd30f52583124543b5d616ef94b31831 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:02 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:14.163069 11776 raft_consensus.cc:504] T cd30f52583124543b5d616ef94b31831 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:14.164098 11822 leader_election.cc:288] T 675b9e33edcf4d4ea7fa428b476e3945 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Requested vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
W20260924 00:21:14.166326 10733 leader_election.cc:341] T 675b9e33edcf4d4ea7fa428b476e3945 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: 675b9e33edcf4d4ea7fa428b476e3945
I20260924 00:21:14.073151 11771 raft_consensus.cc:2726] T 10aa64ed664f40629f5f134052df8f5f P 48fbd603abe146c585d15196e1b5be08 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 00:21:14.164795 11776 leader_election.cc:288] T cd30f52583124543b5d616ef94b31831 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:14.168745 10636 tablet_service.cc:1514] Processing DeleteTablet for tablet 85a0ad61b7cb4e11b30e84ddc4655b64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:13 UTC) from {username='slave'} at 127.0.0.1:46896
W20260924 00:21:14.169560 10732 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:41301, user_credentials={real_user=slave}} blocked reactor thread for 125545us
I20260924 00:21:14.171555 10633 tablet_service.cc:1514] Processing DeleteTablet for tablet 3dae8b8d7743416983e9e903ea57140d with delete_type TABLET_DATA_DELETED (replaced by tablet 08929edc5c5b4ecd80cef4fc472a9ffa) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:14.173828 10367 master_service.cc:946] ReplaceTablet: received request to replace tablet 52c46283934a4c2180f15c7fc38b91a9 from {username='slave'} at 127.0.0.1:47022
I20260924 00:21:14.155865 11835 tablet.cc:1722] T 4e9f00abffe14046bf47a6cae7010690 P c8626e300d0644b0908797ad61bd7cef: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260924 00:21:14.168099 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd30f52583124543b5d616ef94b31831" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
W20260924 00:21:14.174824 10589 leader_election.cc:341] T cd30f52583124543b5d616ef94b31831 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:14.184655 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "675b9e33edcf4d4ea7fa428b476e3945" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef"
I20260924 00:21:14.185273 10942 raft_consensus.cc:3037] T 675b9e33edcf4d4ea7fa428b476e3945 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:14.186733 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c2a729a802e4abe9ab4f989a40198be" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:14.187640 11832 tablet.cc:1722] T 72d3bb80e1034881b94eb75de5d900c2 P b10a9082a37b46f09d581944914c1670: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260924 00:21:14.192250 10732 consensus_peers.cc:487] T cb51d5c6460049108cd7e1974821e578 P 48fbd603abe146c585d15196e1b5be08 -> Peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Couldn't send request to peer c8626e300d0644b0908797ad61bd7cef. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260924 00:21:14.195072 10367 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: Sending DeleteTablet for 3 replicas of tablet 52c46283934a4c2180f15c7fc38b91a9
I20260924 00:21:14.196870 10367 catalog_manager.cc:6565] ReplaceTablet: tablet 52c46283934a4c2180f15c7fc38b91a9 deleted and replaced by tablet a6f54d92b6b047518389615308f0d57c
W20260924 00:21:14.187939 10592 leader_election.cc:341] T 98926dd3850d40938b67779d5fd9a1db P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:14.210283 10592 leader_election.cc:302] T 98926dd3850d40938b67779d5fd9a1db P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:14.210676 10920 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b8d74761b0b4fd0a22021d3870ec05c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:07 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:14.212097 11777 raft_consensus.cc:2726] T 98926dd3850d40938b67779d5fd9a1db P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260924 00:21:14.212165 10592 leader_election.cc:341] T 717759eb61674c89b05b8d2b0dc34979 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:14.212527 10592 leader_election.cc:302] T 717759eb61674c89b05b8d2b0dc34979 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
W20260924 00:21:14.214077 10592 raft_consensus.cc:2670] T 717759eb61674c89b05b8d2b0dc34979 P b10a9082a37b46f09d581944914c1670: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260924 00:21:14.215516 10592 leader_election.cc:341] T 4c2a729a802e4abe9ab4f989a40198be P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:14.215938 10592 leader_election.cc:302] T 4c2a729a802e4abe9ab4f989a40198be P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:14.217028 11776 raft_consensus.cc:2726] T 4c2a729a802e4abe9ab4f989a40198be P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:14.197796 10627 tablet_service.cc:1514] Processing DeleteTablet for tablet 52c46283934a4c2180f15c7fc38b91a9 with delete_type TABLET_DATA_DELETED (replaced by tablet a6f54d92b6b047518389615308f0d57c) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:14.217821 11039 raft_consensus.cc:348] T cd30f52583124543b5d616ef94b31831 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:14.218565 11039 raft_consensus.cc:374] T cd30f52583124543b5d616ef94b31831 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:14.218817 11039 raft_consensus.cc:729] T cd30f52583124543b5d616ef94b31831 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:14.219754 11039 consensus_queue.cc:260] T cd30f52583124543b5d616ef94b31831 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:14.216414 10795 raft_consensus.cc:1226] T 10aa64ed664f40629f5f134052df8f5f P 48fbd603abe146c585d15196e1b5be08 [term 2 FOLLOWER]: Rejecting Update request from peer c8626e300d0644b0908797ad61bd7cef for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260924 00:21:14.228379 10627 tablet_service.cc:1467] Processing CreateTablet for tablet abcc79f486894131957e0895fec6d4f8 (DEFAULT_TABLE table=default.Table_d02a9cefdc804218a76ef33174772e09 [id=9037f3cc30d440c39da4db959a76b52f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:14.229859 10627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abcc79f486894131957e0895fec6d4f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:14.229828 10920 tablet_service.cc:1514] Processing DeleteTablet for tablet a62b85ebb2b84ae19cad3e3e2dd07a59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:09 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:14.239297 10367 catalog_manager.cc:6040] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: default.Table_79486769678c44fe98623f5aa06975bb [id=423a3ef82b0f488fa8fb082d4e6f63a3] alter complete (version 3)
I20260924 00:21:14.247186 11717 ts_tablet_manager.cc:1918] T cb51d5c6460049108cd7e1974821e578 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:14.247622 11717 log.cc:1196] T cb51d5c6460049108cd7e1974821e578 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/cb51d5c6460049108cd7e1974821e578
I20260924 00:21:14.248238 11717 ts_tablet_manager.cc:1939] T cb51d5c6460049108cd7e1974821e578 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:14.252702 11717 tablet_replica.cc:330] T 10aa64ed664f40629f5f134052df8f5f P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:14.221890 11716 ts_tablet_manager.cc:1918] T 0c583d9ba1414f8ea2e605b4b6b076f8 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:14.222004 11040 tablet_bootstrap.cc:492] T 3c850d0474414d2486cd7bbd6568e936 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:14.253459 11040 ts_tablet_manager.cc:1397] T 3c850d0474414d2486cd7bbd6568e936 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.467s user 0.045s sys 0.097s
I20260924 00:21:14.254489 11717 raft_consensus.cc:2229] T 10aa64ed664f40629f5f134052df8f5f P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:21:14.255242 11717 raft_consensus.cc:2258] T 10aa64ed664f40629f5f134052df8f5f P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:14.258285 11838 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b5aa3debe1cd45e19a6bbd0070913ac3)
I20260924 00:21:14.260246 11717 ts_tablet_manager.cc:1905] T 10aa64ed664f40629f5f134052df8f5f P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:14.261216 11040 raft_consensus.cc:348] T 3c850d0474414d2486cd7bbd6568e936 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:14.261919 11040 raft_consensus.cc:374] T 3c850d0474414d2486cd7bbd6568e936 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:14.262154 11040 raft_consensus.cc:729] T 3c850d0474414d2486cd7bbd6568e936 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:14.263695 11040 consensus_queue.cc:260] T 3c850d0474414d2486cd7bbd6568e936 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:14.265637 11040 ts_tablet_manager.cc:1428] T 3c850d0474414d2486cd7bbd6568e936 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.012s user 0.005s sys 0.000s
I20260924 00:21:14.268016 11040 tablet_bootstrap.cc:492] T 5b43ce1fed0749b8b076b9b026914e07 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:14.274224 11041 tablet_bootstrap.cc:654] T 2e839fd66fed4472a0892baa9aa85c54 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:14.283118 10633 tablet_service.cc:1467] Processing CreateTablet for tablet 21d8bbcdcbff4a1abbeb9dba1a62883e (DEFAULT_TABLE table=default.Table_aa256627fd99496e988fab956cde2057 [id=579f89b1e70f4dc2b2080868adfeea76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:14.284132 10633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21d8bbcdcbff4a1abbeb9dba1a62883e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:14.291159 11838 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } }
I20260924 00:21:14.298866 11841 tablet.cc:1722] T 4e9f00abffe14046bf47a6cae7010690 P 48fbd603abe146c585d15196e1b5be08: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260924 00:21:14.335211 11838 leader_election.cc:288] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1d68651913494ce8afee3c136b2da9cc (127.10.4.126:34479), b5aa3debe1cd45e19a6bbd0070913ac3 (127.10.4.125:34487)
I20260924 00:21:14.335999 10309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" candidate_term: 3 candidate_status { last_received { term: 2 index: 322 } } ignore_live_leader: false dest_uuid: "1d68651913494ce8afee3c136b2da9cc" is_pre_election: true
I20260924 00:21:14.337591 10384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" candidate_term: 3 candidate_status { last_received { term: 2 index: 322 } } ignore_live_leader: false dest_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" is_pre_election: true
I20260924 00:21:14.339036 11412 leader_election.cc:302] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f8bd0b3184f84f5f8da10c9c21472f92; no voters: 1d68651913494ce8afee3c136b2da9cc, b5aa3debe1cd45e19a6bbd0070913ac3
I20260924 00:21:14.343312 11710 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 00:21:14.344102 11838 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 00:21:14.306308 10773 tablet_service.cc:1514] Processing DeleteTablet for tablet 85a0ad61b7cb4e11b30e84ddc4655b64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:13 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:14.298058 11716 log.cc:1196] T 0c583d9ba1414f8ea2e605b4b6b076f8 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/0c583d9ba1414f8ea2e605b4b6b076f8
I20260924 00:21:14.375567 11716 ts_tablet_manager.cc:1939] T 0c583d9ba1414f8ea2e605b4b6b076f8 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:14.377702 11716 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35996 (request call id 320) took 1065 ms. Trace:
I20260924 00:21:14.378050 11716 rpcz_store.cc:276] 0924 00:21:13.311831 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:13.314577 (+ 2746us) service_pool.cc:227] Handling call
0924 00:21:13.676151 (+361574us) ts_tablet_manager.cc:1145] Deleting tablet 0c583d9ba1414f8ea2e605b4b6b076f8
0924 00:21:13.676193 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:13.824275 (+148082us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:14.221632 (+397357us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:14.377634 (+156002us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":361049}
I20260924 00:21:14.380657 11716 tablet_replica.cc:330] T 10aa64ed664f40629f5f134052df8f5f P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:14.381273 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 0c583d9ba1414f8ea2e605b4b6b076f8 (table default.Table_fd43d6016c81497586729ff4cbe3213c [id=70e3fa6998b649468ec8575758708401]) successfully deleted
W20260924 00:21:14.381901 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 8e628abffd9545b391c25f83c9903732 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8e628abffd9545b391c25f83c9903732 already in progress: creating tablet
I20260924 00:21:14.383517 11716 raft_consensus.cc:2229] T 10aa64ed664f40629f5f134052df8f5f P 48fbd603abe146c585d15196e1b5be08 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:14.384037 11716 raft_consensus.cc:2258] T 10aa64ed664f40629f5f134052df8f5f P 48fbd603abe146c585d15196e1b5be08 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:14.385792 11716 ts_tablet_manager.cc:1905] T 10aa64ed664f40629f5f134052df8f5f P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:14.308203 10780 tablet_service.cc:1514] Processing DeleteTablet for tablet 3dae8b8d7743416983e9e903ea57140d with delete_type TABLET_DATA_DELETED (replaced by tablet 08929edc5c5b4ecd80cef4fc472a9ffa) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:14.308625 10772 tablet_service.cc:1467] Processing CreateTablet for tablet 36b1bc216f964b898503712eda33eeb6 (DEFAULT_TABLE table=default.Table_eff603db4cdf417c871be7eb94681227 [id=ccd80429571145e0bcb1ade6e3a315d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:14.386920 10772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36b1bc216f964b898503712eda33eeb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:14.315565 11040 tablet_bootstrap.cc:654] T 5b43ce1fed0749b8b076b9b026914e07 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:14.316356 11840 tablet.cc:1722] T 72d3bb80e1034881b94eb75de5d900c2 P 48fbd603abe146c585d15196e1b5be08: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260924 00:21:14.321566 10774 tablet_service.cc:1514] Processing DeleteTablet for tablet 52c46283934a4c2180f15c7fc38b91a9 with delete_type TABLET_DATA_DELETED (replaced by tablet a6f54d92b6b047518389615308f0d57c) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:14.322321 10775 tablet_service.cc:1467] Processing CreateTablet for tablet abcc79f486894131957e0895fec6d4f8 (DEFAULT_TABLE table=default.Table_d02a9cefdc804218a76ef33174772e09 [id=9037f3cc30d440c39da4db959a76b52f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:14.391597 10775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abcc79f486894131957e0895fec6d4f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:14.323699 10776 tablet_service.cc:1467] Processing CreateTablet for tablet 21d8bbcdcbff4a1abbeb9dba1a62883e (DEFAULT_TABLE table=default.Table_aa256627fd99496e988fab956cde2057 [id=579f89b1e70f4dc2b2080868adfeea76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:14.397624 10776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21d8bbcdcbff4a1abbeb9dba1a62883e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:14.324787 10795 raft_consensus.cc:3037] T 3c850d0474414d2486cd7bbd6568e936 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:14.404641 10630 tablet_service.cc:1467] Processing CreateTablet for tablet e880856244d84c18bb09b13566130acb (DEFAULT_TABLE table=default.Table_cbd6d689c0a149d6a22cc163d3efb580 [id=3c20d5d6a1b3460882aa8f3f1bb71184]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:14.325764 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd30f52583124543b5d616ef94b31831" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:14.307513 10777 tablet_service.cc:1467] Processing CreateTablet for tablet 004b12724c404a6fbedfd03fe6da2b2b (DEFAULT_TABLE table=default.Table_e33891e1d1b745819f1114e54bded8bb [id=b382a8277aa3452990df0abe27a80cb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:14.410279 10777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 004b12724c404a6fbedfd03fe6da2b2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:14.416893 10630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e880856244d84c18bb09b13566130acb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:14.376809 11835 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:35358 (request call id 329) took 1055 ms. Trace:
W20260924 00:21:14.443038 10592 leader_election.cc:341] T cd30f52583124543b5d616ef94b31831 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:14.443406 10592 leader_election.cc:302] T cd30f52583124543b5d616ef94b31831 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:14.444058 11777 raft_consensus.cc:2726] T cd30f52583124543b5d616ef94b31831 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:14.369007 11834 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:35358 (request call id 323) took 1165 ms. Trace:
I20260924 00:21:14.297257 11717 ts_tablet_manager.cc:1918] T 10aa64ed664f40629f5f134052df8f5f P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:14.448009 11717 log.cc:1196] T 10aa64ed664f40629f5f134052df8f5f P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/10aa64ed664f40629f5f134052df8f5f
I20260924 00:21:14.449151 11717 ts_tablet_manager.cc:1939] T 10aa64ed664f40629f5f134052df8f5f P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:14.451232 10915 tablet_service.cc:1514] Processing DeleteTablet for tablet 85a0ad61b7cb4e11b30e84ddc4655b64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:13 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:14.453449 10920 tablet_service.cc:1514] Processing DeleteTablet for tablet 52c46283934a4c2180f15c7fc38b91a9 with delete_type TABLET_DATA_DELETED (replaced by tablet a6f54d92b6b047518389615308f0d57c) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:14.453341 10915 tablet_service.cc:1467] Processing CreateTablet for tablet 004b12724c404a6fbedfd03fe6da2b2b (DEFAULT_TABLE table=default.Table_e33891e1d1b745819f1114e54bded8bb [id=b382a8277aa3452990df0abe27a80cb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:14.455015 10915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 004b12724c404a6fbedfd03fe6da2b2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:14.454936 10909 tablet_service.cc:1467] Processing CreateTablet for tablet abcc79f486894131957e0895fec6d4f8 (DEFAULT_TABLE table=default.Table_d02a9cefdc804218a76ef33174772e09 [id=9037f3cc30d440c39da4db959a76b52f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:14.456317 10909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abcc79f486894131957e0895fec6d4f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:14.457500 10367 catalog_manager.cc:5587] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359) does not have the latest schema for tablet 72d3bb80e1034881b94eb75de5d900c2 (table default.Table_b7b4d0037f4946659915a6bd5d65b202 [id=ca942906de0f4019bbdf72ec886fe6fb]). Expected version 3 got 2
I20260924 00:21:14.459080 11777 consensus_queue.cc:1035] T 3c850d0474414d2486cd7bbd6568e936 P b10a9082a37b46f09d581944914c1670 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 00:21:14.460821 10774 tablet_service.cc:1467] Processing CreateTablet for tablet 39cdf419c8b24274bbad1fe4444dd73e (DEFAULT_TABLE table=default.Table_b4a6ce0bb6394e0aadb54d15db1f4b80 [id=02df80a9de5947dbbf8484189637255b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:14.461771 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39cdf419c8b24274bbad1fe4444dd73e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:14.463353 10624 tablet_service.cc:1467] Processing CreateTablet for tablet 39cdf419c8b24274bbad1fe4444dd73e (DEFAULT_TABLE table=default.Table_b4a6ce0bb6394e0aadb54d15db1f4b80 [id=02df80a9de5947dbbf8484189637255b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:14.465003 10624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39cdf419c8b24274bbad1fe4444dd73e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:14.467203 10908 tablet_service.cc:1467] Processing CreateTablet for tablet 21d8bbcdcbff4a1abbeb9dba1a62883e (DEFAULT_TABLE table=default.Table_aa256627fd99496e988fab956cde2057 [id=579f89b1e70f4dc2b2080868adfeea76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:14.446751 11834 rpcz_store.cc:276] 0924 00:21:13.203987 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:13.204083 (+ 96us) service_pool.cc:227] Handling call
0924 00:21:14.368920 (+1164837us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 00:21:13.216815 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0924 00:21:13.217137 (+ 322us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0924 00:21:13.217165 (+ 28us) alter_schema_op.cc:65] Acquired schema lock
0924 00:21:13.217213 (+ 48us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0924 00:21:13.217319 (+ 106us) alter_schema_op.cc:126] START. Timestamp: 1790209273217252
0924 00:21:13.217339 (+ 20us) op_driver.cc:331] REPLICATION: starting
0924 00:21:13.217805 (+ 466us) log.cc:842] Serialized 173 byte log entry
0924 00:21:14.079675 (+861870us) op_driver.cc:444] REPLICATION: finished
0924 00:21:14.147731 (+ 68056us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0924 00:21:14.173860 (+ 26129us) log.cc:842] Serialized 5 byte log entry
0924 00:21:14.304074 (+130214us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:14.368196 (+ 64122us) log.cc:842] Serialized 14 byte log entry
0924 00:21:14.368795 (+ 599us) alter_schema_op.cc:71] Released schema lock
0924 00:21:14.368820 (+ 25us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":67885,"prepare.queue_time_us":12065,"prepare.run_cpu_time_us":18461,"prepare.run_wall_time_us":437720,"raft.queue_time_us":112751,"raft.run_cpu_time_us":1333,"raft.run_wall_time_us":31789,"replication_time_us":862261,"thread_start_us":675718,"threads_started":4,"wal-append.queue_time_us":314546}]]}
I20260924 00:21:14.467701 10910 tablet_service.cc:1467] Processing CreateTablet for tablet e880856244d84c18bb09b13566130acb (DEFAULT_TABLE table=default.Table_cbd6d689c0a149d6a22cc163d3efb580 [id=3c20d5d6a1b3460882aa8f3f1bb71184]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:14.468552 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet cb51d5c6460049108cd7e1974821e578 (table default.Table_9172ff8a00a54ed581e96de0f0b44c3c [id=ed975cbe1bb04e3194882bed0f4ec0b1]) successfully deleted
I20260924 00:21:14.469100 10910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e880856244d84c18bb09b13566130acb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:14.469012 10912 tablet_service.cc:1514] Processing DeleteTablet for tablet 3dae8b8d7743416983e9e903ea57140d with delete_type TABLET_DATA_DELETED (replaced by tablet 08929edc5c5b4ecd80cef4fc472a9ffa) from {username='slave'} at 127.0.0.1:35358
W20260924 00:21:14.469215 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 8e628abffd9545b391c25f83c9903732 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8e628abffd9545b391c25f83c9903732 already in progress: creating tablet
I20260924 00:21:14.470072 10920 tablet_service.cc:1467] Processing CreateTablet for tablet 39cdf419c8b24274bbad1fe4444dd73e (DEFAULT_TABLE table=default.Table_b4a6ce0bb6394e0aadb54d15db1f4b80 [id=02df80a9de5947dbbf8484189637255b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:14.470402 10908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21d8bbcdcbff4a1abbeb9dba1a62883e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:14.471181 10920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39cdf419c8b24274bbad1fe4444dd73e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:14.474788 11801 consensus_queue.cc:1035] T cf36d53828eb421d87b52b3e4cb1d748 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Connected to new peer: Peer: permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 00:21:14.278585 11039 ts_tablet_manager.cc:1428] T cd30f52583124543b5d616ef94b31831 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.214s user 0.001s sys 0.003s
I20260924 00:21:14.478400 11039 tablet_bootstrap.cc:492] T 5107b763acf04df88553ba0af2757570 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:14.260450 10942 raft_consensus.cc:2454] T 675b9e33edcf4d4ea7fa428b476e3945 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 1.
I20260924 00:21:14.482251 11777 consensus_queue.cc:1035] T 3c850d0474414d2486cd7bbd6568e936 P b10a9082a37b46f09d581944914c1670 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:14.482360 11834 tablet.cc:1722] T 72d3bb80e1034881b94eb75de5d900c2 P c8626e300d0644b0908797ad61bd7cef: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260924 00:21:14.440435 11835 rpcz_store.cc:276] 0924 00:21:13.321893 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:13.407394 (+ 85501us) service_pool.cc:227] Handling call
0924 00:21:14.376737 (+969343us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 00:21:13.412505 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0924 00:21:13.412852 (+ 347us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0924 00:21:13.412880 (+ 28us) alter_schema_op.cc:65] Acquired schema lock
0924 00:21:13.412954 (+ 74us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0924 00:21:13.413053 (+ 99us) alter_schema_op.cc:126] START. Timestamp: 1790209273412969
0924 00:21:13.413074 (+ 21us) op_driver.cc:331] REPLICATION: starting
0924 00:21:13.413595 (+ 521us) log.cc:842] Serialized 173 byte log entry
0924 00:21:13.638888 (+225293us) spinlock_profiling.cc:243] Waited 20.8 ms on lock 0x7b540016aba0. stack: 00007f8d51e94033 00007f8d51e93f30 00007f8d51a15534 00000000004db2b4 00000000004db249 00007f8d58da0471 00007f8d58db1c33 00007f8d58dc2288 00007f8d58dc2229 00007f8d58dc21b9 00007f8d58dc2181 00007f8d58dc147d 0000000000580784 00000000005805b9 00007f8d51ec66b6 00007f8d51ec9b11
0924 00:21:14.080799 (+441911us) op_driver.cc:444] REPLICATION: finished
0924 00:21:14.155781 (+ 74982us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0924 00:21:14.175898 (+ 20117us) log.cc:842] Serialized 5 byte log entry
0924 00:21:14.311194 (+135296us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:14.376541 (+ 65347us) log.cc:842] Serialized 14 byte log entry
0924 00:21:14.376620 (+ 79us) alter_schema_op.cc:71] Released schema lock
0924 00:21:14.376643 (+ 23us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":74827,"mutex_wait_us":64460,"prepare.queue_time_us":96,"prepare.run_cpu_time_us":1338,"prepare.run_wall_time_us":1338,"raft.queue_time_us":77744,"raft.run_cpu_time_us":10314,"raft.run_wall_time_us":203477,"replication_time_us":667659,"spinlock_wait_cycles":47848192,"thread_start_us":166962,"threads_started":2}]]}
I20260924 00:21:14.486603 10732 leader_election.cc:302] T 675b9e33edcf4d4ea7fa428b476e3945 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters: b10a9082a37b46f09d581944914c1670
I20260924 00:21:14.487735 11771 raft_consensus.cc:2781] T 675b9e33edcf4d4ea7fa428b476e3945 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:14.488121 11771 raft_consensus.cc:686] T 675b9e33edcf4d4ea7fa428b476e3945 P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: Becoming Leader. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Running, Role: LEADER
I20260924 00:21:14.488830 11771 consensus_queue.cc:237] T 675b9e33edcf4d4ea7fa428b476e3945 P 48fbd603abe146c585d15196e1b5be08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:14.514165 10780 tablet_service.cc:1467] Processing CreateTablet for tablet e880856244d84c18bb09b13566130acb (DEFAULT_TABLE table=default.Table_cbd6d689c0a149d6a22cc163d3efb580 [id=3c20d5d6a1b3460882aa8f3f1bb71184]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:14.515086 10780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e880856244d84c18bb09b13566130acb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:14.514753 10367 catalog_manager.cc:5495] T 5b43ce1fed0749b8b076b9b026914e07 P c8626e300d0644b0908797ad61bd7cef reported cstate change: term changed from 0 to 1, leader changed from <none> to c8626e300d0644b0908797ad61bd7cef (127.10.4.67). New cstate: current_term: 1 leader_uuid: "c8626e300d0644b0908797ad61bd7cef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } health_report { overall_health: HEALTHY } } }
W20260924 00:21:14.524509 10878 consensus_peers.cc:487] T cf36d53828eb421d87b52b3e4cb1d748 P c8626e300d0644b0908797ad61bd7cef -> Peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Couldn't send request to peer 48fbd603abe146c585d15196e1b5be08. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260924 00:21:14.526247 11717 tablet_replica.cc:330] T 675b9e33edcf4d4ea7fa428b476e3945 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:14.527006 11834 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:35358 (request call id 328) took 1229 ms. Trace:
I20260924 00:21:14.527338 11834 rpcz_store.cc:276] 0924 00:21:13.297839 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:13.301018 (+ 3179us) service_pool.cc:227] Handling call
0924 00:21:14.526918 (+1225900us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 00:21:13.654647 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0924 00:21:13.654971 (+ 324us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0924 00:21:14.369048 (+714077us) alter_schema_op.cc:65] Acquired schema lock
0924 00:21:14.369095 (+ 47us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0924 00:21:14.369193 (+ 98us) alter_schema_op.cc:126] START. Timestamp: 1790209274369080
0924 00:21:14.369213 (+ 20us) op_driver.cc:331] REPLICATION: starting
0924 00:21:14.369657 (+ 444us) log.cc:842] Serialized 173 byte log entry
0924 00:21:14.448841 (+ 79184us) op_driver.cc:444] REPLICATION: finished
0924 00:21:14.482222 (+ 33381us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0924 00:21:14.483568 (+ 1346us) log.cc:842] Serialized 5 byte log entry
0924 00:21:14.495926 (+ 12358us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:14.513635 (+ 17709us) log.cc:842] Serialized 14 byte log entry
0924 00:21:14.520824 (+ 7189us) alter_schema_op.cc:71] Released schema lock
0924 00:21:14.522789 (+ 1965us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":5640,"prepare.queue_time_us":352929,"prepare.run_cpu_time_us":30668,"prepare.run_wall_time_us":716139,"raft.queue_time_us":101,"raft.run_cpu_time_us":547,"raft.run_wall_time_us":546,"replication_time_us":79383,"spinlock_wait_cycles":59648,"threads_started":1}]]}
I20260924 00:21:14.536078 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 10aa64ed664f40629f5f134052df8f5f (table default.Table_d56dcf7ebba04b769e2e233cc7d9fa0e [id=389f9c4d091d4c1da824aafd4f9ebb2b]) successfully deleted
I20260924 00:21:14.513767 11799 raft_consensus.cc:482] T cd30f52583124543b5d616ef94b31831 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:14.543391 11799 raft_consensus.cc:504] T cd30f52583124543b5d616ef94b31831 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:14.545621 11799 leader_election.cc:288] T cd30f52583124543b5d616ef94b31831 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:14.547468 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd30f52583124543b5d616ef94b31831" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:14.548029 10654 raft_consensus.cc:2454] T cd30f52583124543b5d616ef94b31831 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
I20260924 00:21:14.549893 11717 raft_consensus.cc:2229] T 675b9e33edcf4d4ea7fa428b476e3945 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:14.550261 10876 leader_election.cc:302] T cd30f52583124543b5d616ef94b31831 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:14.550457 11717 raft_consensus.cc:2258] T 675b9e33edcf4d4ea7fa428b476e3945 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:14.551330 11807 raft_consensus.cc:2781] T cd30f52583124543b5d616ef94b31831 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:14.551630 11807 raft_consensus.cc:482] T cd30f52583124543b5d616ef94b31831 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:14.551930 11807 raft_consensus.cc:3037] T cd30f52583124543b5d616ef94b31831 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:14.553025 11717 ts_tablet_manager.cc:1905] T 675b9e33edcf4d4ea7fa428b476e3945 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:14.559312 11807 raft_consensus.cc:504] T cd30f52583124543b5d616ef94b31831 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:14.581116 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd30f52583124543b5d616ef94b31831" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:21:14.581612 10654 raft_consensus.cc:3037] T cd30f52583124543b5d616ef94b31831 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:14.583218 10906 tablet_service.cc:1514] Processing DeleteTablet for tablet 10aa64ed664f40629f5f134052df8f5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:13 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:14.587287 10654 raft_consensus.cc:2454] T cd30f52583124543b5d616ef94b31831 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 1.
I20260924 00:21:14.588199 10876 leader_election.cc:302] T cd30f52583124543b5d616ef94b31831 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:14.588532 10915 tablet_service.cc:1467] Processing CreateTablet for tablet a3714466ae1441c2a9e32e26b9c37617 (DEFAULT_TABLE table=default.Table_2ff16b0fac094dc1865b40d6c91bbfdc [id=6fca68e9ecc44d71bae53204e1894542]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:14.588901 11799 raft_consensus.cc:2781] T cd30f52583124543b5d616ef94b31831 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:14.589491 11799 raft_consensus.cc:686] T cd30f52583124543b5d616ef94b31831 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Becoming Leader. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Running, Role: LEADER
I20260924 00:21:14.590422 11832 tablet.cc:1722] T 72d3bb80e1034881b94eb75de5d900c2 P b10a9082a37b46f09d581944914c1670: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260924 00:21:14.590444 11799 consensus_queue.cc:237] T cd30f52583124543b5d616ef94b31831 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:14.593924 10915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3714466ae1441c2a9e32e26b9c37617. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:14.602965 11807 leader_election.cc:288] T cd30f52583124543b5d616ef94b31831 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:14.605315 11041 tablet_bootstrap.cc:492] T 2e839fd66fed4472a0892baa9aa85c54 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:14.605707 11041 ts_tablet_manager.cc:1397] T 2e839fd66fed4472a0892baa9aa85c54 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.548s user 0.010s sys 0.000s
I20260924 00:21:14.611181 10633 tablet_service.cc:1467] Processing CreateTablet for tablet a3714466ae1441c2a9e32e26b9c37617 (DEFAULT_TABLE table=default.Table_2ff16b0fac094dc1865b40d6c91bbfdc [id=6fca68e9ecc44d71bae53204e1894542]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:14.612665 10633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3714466ae1441c2a9e32e26b9c37617. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:14.577924 11040 tablet_bootstrap.cc:492] T 5b43ce1fed0749b8b076b9b026914e07 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:14.616710 11040 ts_tablet_manager.cc:1397] T 5b43ce1fed0749b8b076b9b026914e07 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.349s user 0.006s sys 0.007s
I20260924 00:21:14.624768 10794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd30f52583124543b5d616ef94b31831" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:14.626242 10794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd30f52583124543b5d616ef94b31831" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
W20260924 00:21:14.628417 10878 leader_election.cc:341] T cd30f52583124543b5d616ef94b31831 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
W20260924 00:21:14.629338 10878 leader_election.cc:341] T cd30f52583124543b5d616ef94b31831 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:14.629782 10367 catalog_manager.cc:6040] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3: default.Table_b7b4d0037f4946659915a6bd5d65b202 [id=ca942906de0f4019bbdf72ec886fe6fb] alter complete (version 3)
I20260924 00:21:14.618805 11040 raft_consensus.cc:348] T 5b43ce1fed0749b8b076b9b026914e07 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:14.655359 11040 raft_consensus.cc:374] T 5b43ce1fed0749b8b076b9b026914e07 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:14.655639 11040 raft_consensus.cc:729] T 5b43ce1fed0749b8b076b9b026914e07 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:14.656185 11040 consensus_queue.cc:260] T 5b43ce1fed0749b8b076b9b026914e07 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:14.658095 11040 ts_tablet_manager.cc:1428] T 5b43ce1fed0749b8b076b9b026914e07 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.041s user 0.004s sys 0.000s
I20260924 00:21:14.662518 11041 raft_consensus.cc:348] T 2e839fd66fed4472a0892baa9aa85c54 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:14.663211 11041 raft_consensus.cc:374] T 2e839fd66fed4472a0892baa9aa85c54 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:14.653544 10774 tablet_service.cc:1467] Processing CreateTablet for tablet a3714466ae1441c2a9e32e26b9c37617 (DEFAULT_TABLE table=default.Table_2ff16b0fac094dc1865b40d6c91bbfdc [id=6fca68e9ecc44d71bae53204e1894542]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:14.674727 11041 raft_consensus.cc:729] T 2e839fd66fed4472a0892baa9aa85c54 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:14.675209 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3714466ae1441c2a9e32e26b9c37617. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:14.677690 11041 consensus_queue.cc:260] T 2e839fd66fed4472a0892baa9aa85c54 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:14.697218 10909 tablet_service.cc:1514] Processing DeleteTablet for tablet cd30f52583124543b5d616ef94b31831 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:02 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:14.703653 11039 tablet_bootstrap.cc:700] T 5107b763acf04df88553ba0af2757570 P c8626e300d0644b0908797ad61bd7cef: Time spent opening tablet: real 0.129s user 0.002s sys 0.000s
I20260924 00:21:14.707444 11039 tablet_bootstrap.cc:654] T 5107b763acf04df88553ba0af2757570 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:14.725924 11840 tablet.cc:1722] T 72d3bb80e1034881b94eb75de5d900c2 P 48fbd603abe146c585d15196e1b5be08: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260924 00:21:14.726625 11801 consensus_queue.cc:1035] T 5b43ce1fed0749b8b076b9b026914e07 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Connected to new peer: Peer: permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:14.739017 11040 tablet_bootstrap.cc:492] T 717759eb61674c89b05b8d2b0dc34979 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:14.742434 11717 ts_tablet_manager.cc:1918] T 675b9e33edcf4d4ea7fa428b476e3945 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:14.745682 11041 ts_tablet_manager.cc:1428] T 2e839fd66fed4472a0892baa9aa85c54 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.140s user 0.006s sys 0.000s
I20260924 00:21:14.748684 11771 raft_consensus.cc:482] T 5b43ce1fed0749b8b076b9b026914e07 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:14.749248 11771 raft_consensus.cc:504] T 5b43ce1fed0749b8b076b9b026914e07 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:14.749660 11717 log.cc:1196] T 675b9e33edcf4d4ea7fa428b476e3945 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/675b9e33edcf4d4ea7fa428b476e3945
I20260924 00:21:14.750402 11717 ts_tablet_manager.cc:1939] T 675b9e33edcf4d4ea7fa428b476e3945 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:14.752635 11717 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35358 (request call id 340) took 1013 ms. Trace:
I20260924 00:21:14.753094 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b43ce1fed0749b8b076b9b026914e07" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:14.752921 11717 rpcz_store.cc:276] 0924 00:21:13.739479 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:13.739569 (+ 90us) service_pool.cc:227] Handling call
0924 00:21:14.526044 (+786475us) ts_tablet_manager.cc:1145] Deleting tablet 675b9e33edcf4d4ea7fa428b476e3945
0924 00:21:14.526127 (+ 83us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:14.699943 (+173816us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:14.742206 (+ 42263us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:14.752571 (+ 10365us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":14170,"tablet-delete.queue_time_us":785955}
I20260924 00:21:14.754248 11717 tablet_replica.cc:330] T cf36d53828eb421d87b52b3e4cb1d748 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:14.754242 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 675b9e33edcf4d4ea7fa428b476e3945 (table default.Table_ee73693b7a6e46a3af53f7828da00f9e [id=9359b05533064c38b9bd589c0bb06471]) successfully deleted
I20260924 00:21:14.754992 11717 raft_consensus.cc:2229] T cf36d53828eb421d87b52b3e4cb1d748 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:21:14.755817 11717 raft_consensus.cc:2258] T cf36d53828eb421d87b52b3e4cb1d748 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:14.764742 11717 ts_tablet_manager.cc:1905] T cf36d53828eb421d87b52b3e4cb1d748 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:14.770099 11719 ts_tablet_manager.cc:1918] T 10aa64ed664f40629f5f134052df8f5f P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:14.770603 11719 log.cc:1196] T 10aa64ed664f40629f5f134052df8f5f P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/10aa64ed664f40629f5f134052df8f5f
I20260924 00:21:14.771379 11719 ts_tablet_manager.cc:1939] T 10aa64ed664f40629f5f134052df8f5f P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:14.777866 11717 ts_tablet_manager.cc:1918] T cf36d53828eb421d87b52b3e4cb1d748 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:14.778322 11717 log.cc:1196] T cf36d53828eb421d87b52b3e4cb1d748 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/cf36d53828eb421d87b52b3e4cb1d748
I20260924 00:21:14.779078 11717 ts_tablet_manager.cc:1939] T cf36d53828eb421d87b52b3e4cb1d748 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:14.781265 11717 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35358 (request call id 341) took 1035 ms. Trace:
I20260924 00:21:14.781656 11717 rpcz_store.cc:276] 0924 00:21:13.745465 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:13.745557 (+ 92us) service_pool.cc:227] Handling call
0924 00:21:14.754105 (+1008548us) ts_tablet_manager.cc:1145] Deleting tablet cf36d53828eb421d87b52b3e4cb1d748
0924 00:21:14.754156 (+ 51us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:14.771369 (+ 17213us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:14.777645 (+ 6276us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:14.781190 (+ 3545us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1007886}
I20260924 00:21:14.782588 10797 raft_consensus.cc:3037] T 5b43ce1fed0749b8b076b9b026914e07 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:14.783805 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet cf36d53828eb421d87b52b3e4cb1d748 (table default.Table_0cdc660142f44fb5b77963f9432b4182 [id=15ae26a9e629499e9c632282502ed104]) successfully deleted
I20260924 00:21:14.784006 11717 tablet_replica.cc:330] T a62b85ebb2b84ae19cad3e3e2dd07a59 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
W20260924 00:21:14.784397 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 1492ece2346d461ea57096a897daf83f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1492ece2346d461ea57096a897daf83f already in progress: creating tablet
I20260924 00:21:14.784827 11717 raft_consensus.cc:2229] T a62b85ebb2b84ae19cad3e3e2dd07a59 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus shutting down.
W20260924 00:21:14.784931 10358 catalog_manager.cc:4808] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 3b8d74761b0b4fd0a22021d3870ec05c because the tablet was not found. No further retry: Not found: Tablet not found: 3b8d74761b0b4fd0a22021d3870ec05c
I20260924 00:21:14.785303 11717 raft_consensus.cc:2258] T a62b85ebb2b84ae19cad3e3e2dd07a59 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:14.787851 11717 ts_tablet_manager.cc:1905] T a62b85ebb2b84ae19cad3e3e2dd07a59 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:14.790385 10653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b43ce1fed0749b8b076b9b026914e07" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:14.792016 10733 leader_election.cc:302] T 5b43ce1fed0749b8b076b9b026914e07 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08; no voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:14.792856 11822 raft_consensus.cc:2726] T 5b43ce1fed0749b8b076b9b026914e07 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:14.793565 11771 leader_election.cc:288] T 5b43ce1fed0749b8b076b9b026914e07 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
W20260924 00:21:14.795710 10592 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:35359, user_credentials={real_user=slave}} blocked reactor thread for 168078us
I20260924 00:21:14.800609 11808 consensus_queue.cc:1035] T 5b43ce1fed0749b8b076b9b026914e07 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Connected to new peer: Peer: permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260924 00:21:14.800611 11717 ts_tablet_manager.cc:1918] T a62b85ebb2b84ae19cad3e3e2dd07a59 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:14.814955 11717 log.cc:1196] T a62b85ebb2b84ae19cad3e3e2dd07a59 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/a62b85ebb2b84ae19cad3e3e2dd07a59
I20260924 00:21:14.815549 11717 ts_tablet_manager.cc:1939] T a62b85ebb2b84ae19cad3e3e2dd07a59 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:14.817807 11717 tablet_replica.cc:330] T 85a0ad61b7cb4e11b30e84ddc4655b64 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:14.825335 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet a62b85ebb2b84ae19cad3e3e2dd07a59 (table default.Table_3e21dcff4f0b40299e8ddb8ebb7f3c22 [id=3f6c757389bb4213b584f3e5a9ac7ede]) successfully deleted
I20260924 00:21:14.878742 11717 raft_consensus.cc:2229] T 85a0ad61b7cb4e11b30e84ddc4655b64 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:14.879273 11717 raft_consensus.cc:2258] T 85a0ad61b7cb4e11b30e84ddc4655b64 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:14.885599 11719 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 323) took 1136 ms. Trace:
I20260924 00:21:14.886021 11719 rpcz_store.cc:276] 0924 00:21:13.749640 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:13.750355 (+ 715us) service_pool.cc:227] Handling call
0924 00:21:14.028409 (+278054us) ts_tablet_manager.cc:1145] Deleting tablet 10aa64ed664f40629f5f134052df8f5f
0924 00:21:14.028447 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:14.040167 (+ 11720us) spinlock_profiling.cc:243] Waited 8.88 ms on lock 0x7b54000c3354. stack: 00007f350003f033 00007f350003ef30 00007f34ffbc0534 00000000004db2b4 00000000004db249 00007f3506f4b471 00007f3506fce352 00007f35073e426c 00007f3507ff1cfa 00007f3508021e10 00007f3508008e0d 00007f3508008db9 00007f3508008d49 00007f3508008d11 00007f3508007f3d 0000000000580784
0924 00:21:14.315415 (+275248us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:14.769857 (+454442us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:14.885533 (+115676us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":20430080,"tablet-delete.queue_time_us":234288}
I20260924 00:21:14.887607 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 10aa64ed664f40629f5f134052df8f5f (table default.Table_d56dcf7ebba04b769e2e233cc7d9fa0e [id=389f9c4d091d4c1da824aafd4f9ebb2b]) successfully deleted
I20260924 00:21:14.898239 11717 ts_tablet_manager.cc:1905] T 85a0ad61b7cb4e11b30e84ddc4655b64 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260924 00:21:14.909183 10592 outbound_call.cc:322] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.10.4.124:35345, user_credentials={real_user=slave}} blocked reactor thread for 67724.3us
I20260924 00:21:14.920115 11719 tablet_replica.cc:330] T cf36d53828eb421d87b52b3e4cb1d748 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:14.941045 11041 tablet_bootstrap.cc:492] T a53f06276dbb4d9f97a154a8ceb0278d P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:14.946806 11039 tablet_bootstrap.cc:492] T 5107b763acf04df88553ba0af2757570 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:14.947438 11039 ts_tablet_manager.cc:1397] T 5107b763acf04df88553ba0af2757570 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.469s user 0.010s sys 0.000s
I20260924 00:21:14.949728 11039 raft_consensus.cc:348] T 5107b763acf04df88553ba0af2757570 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:14.950423 11039 raft_consensus.cc:374] T 5107b763acf04df88553ba0af2757570 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:14.950707 11039 raft_consensus.cc:729] T 5107b763acf04df88553ba0af2757570 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:14.951321 11039 consensus_queue.cc:260] T 5107b763acf04df88553ba0af2757570 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:14.959348 11040 tablet_bootstrap.cc:654] T 717759eb61674c89b05b8d2b0dc34979 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:14.969444 11719 raft_consensus.cc:2229] T cf36d53828eb421d87b52b3e4cb1d748 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:14.970058 11719 raft_consensus.cc:2258] T cf36d53828eb421d87b52b3e4cb1d748 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:14.981365 11717 ts_tablet_manager.cc:1918] T 85a0ad61b7cb4e11b30e84ddc4655b64 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260924 00:21:14.981750 11717 log.cc:1196] T 85a0ad61b7cb4e11b30e84ddc4655b64 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/85a0ad61b7cb4e11b30e84ddc4655b64
I20260924 00:21:14.982393 11717 ts_tablet_manager.cc:1939] T 85a0ad61b7cb4e11b30e84ddc4655b64 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:14.990597 11716 ts_tablet_manager.cc:1918] T 10aa64ed664f40629f5f134052df8f5f P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:14.997818 11716 log.cc:1196] T 10aa64ed664f40629f5f134052df8f5f P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/10aa64ed664f40629f5f134052df8f5f
I20260924 00:21:14.999631 11716 ts_tablet_manager.cc:1939] T 10aa64ed664f40629f5f134052df8f5f P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:15.001972 11041 tablet_bootstrap.cc:654] T a53f06276dbb4d9f97a154a8ceb0278d P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:15.004391 11719 ts_tablet_manager.cc:1905] T cf36d53828eb421d87b52b3e4cb1d748 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:15.014838 11039 ts_tablet_manager.cc:1428] T 5107b763acf04df88553ba0af2757570 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.067s user 0.005s sys 0.000s
I20260924 00:21:15.015579 10908 tablet_service.cc:1514] Processing DeleteTablet for tablet cf36d53828eb421d87b52b3e4cb1d748 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:05 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:15.022836 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 85a0ad61b7cb4e11b30e84ddc4655b64 (table default.Table_07390b277c404c7c9daecd17fe3dccca [id=12285914811f4220822b47a2a62b01ab]) successfully deleted
W20260924 00:21:15.040827 10589 consensus_peers.cc:487] T 85a0ad61b7cb4e11b30e84ddc4655b64 P b10a9082a37b46f09d581944914c1670 -> Peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Couldn't send request to peer c8626e300d0644b0908797ad61bd7cef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 85a0ad61b7cb4e11b30e84ddc4655b64. This is attempt 1: this message will repeat every 5th retry.
I20260924 00:21:15.045364 11039 tablet_bootstrap.cc:492] T 9319f9caadb04067a2e9700b5019ed37 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:15.058547 11039 tablet_bootstrap.cc:654] T 9319f9caadb04067a2e9700b5019ed37 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:15.069846 11717 tablet_replica.cc:330] T cd30f52583124543b5d616ef94b31831 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:15.070585 11717 raft_consensus.cc:2229] T cd30f52583124543b5d616ef94b31831 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:21:15.071208 11717 pending_rounds.cc:62] T cd30f52583124543b5d616ef94b31831 P c8626e300d0644b0908797ad61bd7cef: Trying to abort 1 pending ops.
I20260924 00:21:15.071419 11717 pending_rounds.cc:68] T cd30f52583124543b5d616ef94b31831 P c8626e300d0644b0908797ad61bd7cef: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332697188731318272 op_type: NO_OP noop_request { }
I20260924 00:21:15.071800 11717 raft_consensus.cc:2866] T cd30f52583124543b5d616ef94b31831 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260924 00:21:15.072141 11717 raft_consensus.cc:2258] T cd30f52583124543b5d616ef94b31831 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 00:21:15.072913 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 52c46283934a4c2180f15c7fc38b91a9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 52c46283934a4c2180f15c7fc38b91a9 already in progress: creating tablet
W20260924 00:21:15.073603 10358 catalog_manager.cc:4821] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 3dae8b8d7743416983e9e903ea57140d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3dae8b8d7743416983e9e903ea57140d already in progress: creating tablet
W20260924 00:21:15.074144 10358 catalog_manager.cc:4808] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 10aa64ed664f40629f5f134052df8f5f because the tablet was not found. No further retry: Not found: Tablet not found: 10aa64ed664f40629f5f134052df8f5f
I20260924 00:21:15.075110 11717 ts_tablet_manager.cc:1905] T cd30f52583124543b5d616ef94b31831 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:15.090442 11039 tablet_bootstrap.cc:492] T 9319f9caadb04067a2e9700b5019ed37 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:15.090947 11039 ts_tablet_manager.cc:1397] T 9319f9caadb04067a2e9700b5019ed37 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.046s user 0.005s sys 0.005s
I20260924 00:21:15.093237 11039 raft_consensus.cc:348] T 9319f9caadb04067a2e9700b5019ed37 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:15.094027 11039 raft_consensus.cc:374] T 9319f9caadb04067a2e9700b5019ed37 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:15.094318 11039 raft_consensus.cc:729] T 9319f9caadb04067a2e9700b5019ed37 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:15.094996 11039 consensus_queue.cc:260] T 9319f9caadb04067a2e9700b5019ed37 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
W20260924 00:21:15.096040 10360 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 4c83e8c5109b43fb8268ec3ed52199be (table default.Table_83435d6ddd64425390ab1b05337d2081 [id=bf5691bc3ec148eda2cb116bb7842cfc], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 4c83e8c5109b43fb8268ec3ed52199be consensus state has no leader
I20260924 00:21:15.096906 11039 ts_tablet_manager.cc:1428] T 9319f9caadb04067a2e9700b5019ed37 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260924 00:21:15.097723 11039 tablet_bootstrap.cc:492] T 98926dd3850d40938b67779d5fd9a1db P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:15.102455 11039 tablet_bootstrap.cc:654] T 98926dd3850d40938b67779d5fd9a1db P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:15.117884 10367 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 6bcfd729e9bd4040b3384ab24164e80d (table default.Table_9670783a92074b639ecb0d6c374811e5 [id=df2a5a74db65431c9ecb750184f64709]). Expected version 1 got 0
I20260924 00:21:15.120395 11040 tablet_bootstrap.cc:492] T 717759eb61674c89b05b8d2b0dc34979 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:15.120898 11040 ts_tablet_manager.cc:1397] T 717759eb61674c89b05b8d2b0dc34979 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.382s user 0.008s sys 0.001s
I20260924 00:21:15.123238 11040 raft_consensus.cc:348] T 717759eb61674c89b05b8d2b0dc34979 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:15.134171 11717 ts_tablet_manager.cc:1918] T cd30f52583124543b5d616ef94b31831 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:15.134879 11717 log.cc:1196] T cd30f52583124543b5d616ef94b31831 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/cd30f52583124543b5d616ef94b31831
I20260924 00:21:15.135566 11717 ts_tablet_manager.cc:1939] T cd30f52583124543b5d616ef94b31831 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:15.137377 11039 tablet_bootstrap.cc:492] T 98926dd3850d40938b67779d5fd9a1db P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:15.138945 11039 ts_tablet_manager.cc:1397] T 98926dd3850d40938b67779d5fd9a1db P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.041s user 0.008s sys 0.001s
I20260924 00:21:15.145887 11039 raft_consensus.cc:348] T 98926dd3850d40938b67779d5fd9a1db P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:15.146495 11039 raft_consensus.cc:374] T 98926dd3850d40938b67779d5fd9a1db P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:15.146716 11039 raft_consensus.cc:729] T 98926dd3850d40938b67779d5fd9a1db P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:15.149540 11039 consensus_queue.cc:260] T 98926dd3850d40938b67779d5fd9a1db P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:15.151829 11041 tablet_bootstrap.cc:492] T a53f06276dbb4d9f97a154a8ceb0278d P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:15.137858 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet cd30f52583124543b5d616ef94b31831 (table default.Table_3078c59b2af247ab91ea35e4f8bf9b9c [id=89a81661022b40eda6eecd81008a3107]) successfully deleted
W20260924 00:21:15.157413 10358 catalog_manager.cc:4808] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet cf36d53828eb421d87b52b3e4cb1d748 because the tablet was not found. No further retry: Not found: Tablet not found: cf36d53828eb421d87b52b3e4cb1d748
I20260924 00:21:15.161181 11041 ts_tablet_manager.cc:1397] T a53f06276dbb4d9f97a154a8ceb0278d P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.415s user 0.009s sys 0.000s
I20260924 00:21:15.163228 11041 raft_consensus.cc:348] T a53f06276dbb4d9f97a154a8ceb0278d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:15.166165 11039 ts_tablet_manager.cc:1428] T 98926dd3850d40938b67779d5fd9a1db P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.027s user 0.005s sys 0.000s
I20260924 00:21:15.167697 11039 tablet_bootstrap.cc:492] T 57ae3d10846a4aaa84eba81b13aafd44 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:15.179747 10908 tablet_service.cc:1514] Processing DeleteTablet for tablet 5107b763acf04df88553ba0af2757570 with delete_type TABLET_DATA_DELETED (replaced by tablet cf0f0f016d074b1fb78d9b427c67fe86) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:15.180388 11717 tablet_replica.cc:330] T 5107b763acf04df88553ba0af2757570 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:15.181089 11717 raft_consensus.cc:2229] T 5107b763acf04df88553ba0af2757570 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:15.181495 11717 raft_consensus.cc:2258] T 5107b763acf04df88553ba0af2757570 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:15.183245 11717 ts_tablet_manager.cc:1905] T 5107b763acf04df88553ba0af2757570 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:15.098156 11716 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35996 (request call id 326) took 1444 ms. Trace:
I20260924 00:21:15.200493 11716 rpcz_store.cc:276] 0924 00:21:13.653637 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:13.653741 (+ 104us) service_pool.cc:227] Handling call
0924 00:21:14.380518 (+726777us) ts_tablet_manager.cc:1145] Deleting tablet 10aa64ed664f40629f5f134052df8f5f
0924 00:21:14.380553 (+ 35us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:14.630955 (+250402us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:14.990343 (+359388us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:15.098088 (+107745us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":726229}
I20260924 00:21:15.201987 11716 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35996 (request call id 327) took 1299 ms. Trace:
I20260924 00:21:15.201988 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 10aa64ed664f40629f5f134052df8f5f (table default.Table_d56dcf7ebba04b769e2e233cc7d9fa0e [id=389f9c4d091d4c1da824aafd4f9ebb2b]) successfully deleted
I20260924 00:21:15.202369 11716 rpcz_store.cc:276] 0924 00:21:13.902433 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:13.903260 (+ 827us) service_pool.cc:227] Handling call
0924 00:21:15.201719 (+1298459us) ts_tablet_manager.cc:1145] Deleting tablet a0f71049f6b3478da3b07ebc9293ce64
0924 00:21:15.201763 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:15.201926 (+ 163us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1297919}
W20260924 00:21:15.203480 10361 catalog_manager.cc:4808] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet a0f71049f6b3478da3b07ebc9293ce64 because the tablet was not found. No further retry: Not found: Tablet not found: a0f71049f6b3478da3b07ebc9293ce64
I20260924 00:21:15.203732 11716 tablet_replica.cc:330] T cb51d5c6460049108cd7e1974821e578 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:15.204438 11716 raft_consensus.cc:2229] T cb51d5c6460049108cd7e1974821e578 P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:21:15.205389 11716 pending_rounds.cc:62] T cb51d5c6460049108cd7e1974821e578 P 48fbd603abe146c585d15196e1b5be08: Trying to abort 1 pending ops.
I20260924 00:21:15.205829 11716 pending_rounds.cc:68] T cb51d5c6460049108cd7e1974821e578 P 48fbd603abe146c585d15196e1b5be08: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332697182449057792 op_type: NO_OP noop_request { }
I20260924 00:21:15.206297 11716 raft_consensus.cc:2866] T cb51d5c6460049108cd7e1974821e578 P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260924 00:21:15.206676 11716 raft_consensus.cc:2258] T cb51d5c6460049108cd7e1974821e578 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:15.208714 11716 ts_tablet_manager.cc:1905] T cb51d5c6460049108cd7e1974821e578 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:15.209034 11040 raft_consensus.cc:374] T 717759eb61674c89b05b8d2b0dc34979 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:15.209362 11040 raft_consensus.cc:729] T 717759eb61674c89b05b8d2b0dc34979 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:15.209996 11040 consensus_queue.cc:260] T 717759eb61674c89b05b8d2b0dc34979 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:15.218660 11717 ts_tablet_manager.cc:1918] T 5107b763acf04df88553ba0af2757570 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:15.218974 11717 log.cc:1196] T 5107b763acf04df88553ba0af2757570 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/5107b763acf04df88553ba0af2757570
I20260924 00:21:15.219532 11717 ts_tablet_manager.cc:1939] T 5107b763acf04df88553ba0af2757570 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:15.224156 11854 raft_consensus.cc:482] T 2e839fd66fed4472a0892baa9aa85c54 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:15.224593 11854 raft_consensus.cc:504] T 2e839fd66fed4472a0892baa9aa85c54 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:15.226980 11854 leader_election.cc:288] T 2e839fd66fed4472a0892baa9aa85c54 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:15.227953 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e839fd66fed4472a0892baa9aa85c54" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:15.228242 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e839fd66fed4472a0892baa9aa85c54" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:15.229202 10592 leader_election.cc:341] T 2e839fd66fed4472a0892baa9aa85c54 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
W20260924 00:21:15.229743 10589 leader_election.cc:341] T 2e839fd66fed4472a0892baa9aa85c54 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:15.230058 10589 leader_election.cc:302] T 2e839fd66fed4472a0892baa9aa85c54 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:15.233114 11854 raft_consensus.cc:2726] T 2e839fd66fed4472a0892baa9aa85c54 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:15.163895 11041 raft_consensus.cc:374] T a53f06276dbb4d9f97a154a8ceb0278d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:15.239187 11041 raft_consensus.cc:729] T a53f06276dbb4d9f97a154a8ceb0278d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:15.239987 11041 consensus_queue.cc:260] T a53f06276dbb4d9f97a154a8ceb0278d P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:15.243225 11041 ts_tablet_manager.cc:1428] T a53f06276dbb4d9f97a154a8ceb0278d P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.082s user 0.005s sys 0.000s
I20260924 00:21:15.246737 11039 tablet_bootstrap.cc:654] T 57ae3d10846a4aaa84eba81b13aafd44 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:15.249061 11040 ts_tablet_manager.cc:1428] T 717759eb61674c89b05b8d2b0dc34979 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.128s user 0.005s sys 0.000s
I20260924 00:21:15.288919 11041 tablet_bootstrap.cc:492] T d4e120acff3e4c30aed813b0b24fbfee P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:15.294147 10624 tablet_service.cc:1514] Processing DeleteTablet for tablet a53f06276dbb4d9f97a154a8ceb0278d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:05 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:15.295276 11867 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b5aa3debe1cd45e19a6bbd0070913ac3)
I20260924 00:21:15.295759 11867 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } }
I20260924 00:21:15.311092 11041 tablet_bootstrap.cc:654] T d4e120acff3e4c30aed813b0b24fbfee P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:15.311379 10384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1d68651913494ce8afee3c136b2da9cc" candidate_term: 3 candidate_status { last_received { term: 2 index: 453 } } ignore_live_leader: false dest_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" is_pre_election: true
I20260924 00:21:15.329336 10774 tablet_service.cc:1514] Processing DeleteTablet for tablet 717759eb61674c89b05b8d2b0dc34979 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:03 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:15.332906 11867 leader_election.cc:288] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b5aa3debe1cd45e19a6bbd0070913ac3 (127.10.4.125:34487), f8bd0b3184f84f5f8da10c9c21472f92 (127.10.4.124:35345)
W20260924 00:21:15.334440 10732 consensus_peers.cc:487] T 675b9e33edcf4d4ea7fa428b476e3945 P 48fbd603abe146c585d15196e1b5be08 -> Peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Couldn't send request to peer c8626e300d0644b0908797ad61bd7cef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 675b9e33edcf4d4ea7fa428b476e3945. This is attempt 1: this message will repeat every 5th retry.
I20260924 00:21:15.335358 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 5107b763acf04df88553ba0af2757570 (table default.Table_e412cbac77574c5aafc587eaa698b633 [id=667f4349af3f4eb28461e33f65beeb1c]) successfully deleted
I20260924 00:21:15.343706 11039 tablet_bootstrap.cc:492] T 57ae3d10846a4aaa84eba81b13aafd44 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:15.344089 11039 ts_tablet_manager.cc:1397] T 57ae3d10846a4aaa84eba81b13aafd44 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.177s user 0.023s sys 0.003s
I20260924 00:21:15.346371 11039 raft_consensus.cc:348] T 57ae3d10846a4aaa84eba81b13aafd44 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:15.347134 11039 raft_consensus.cc:374] T 57ae3d10846a4aaa84eba81b13aafd44 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:15.347384 11039 raft_consensus.cc:729] T 57ae3d10846a4aaa84eba81b13aafd44 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:15.348131 11039 consensus_queue.cc:260] T 57ae3d10846a4aaa84eba81b13aafd44 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:15.364548 11039 ts_tablet_manager.cc:1428] T 57ae3d10846a4aaa84eba81b13aafd44 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.020s user 0.005s sys 0.000s
I20260924 00:21:15.373922 11039 tablet_bootstrap.cc:492] T 4c2a729a802e4abe9ab4f989a40198be P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:15.402043 11040 tablet_bootstrap.cc:492] T a62b85ebb2b84ae19cad3e3e2dd07a59 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:15.404148 11039 tablet_bootstrap.cc:654] T 4c2a729a802e4abe9ab4f989a40198be P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:15.429116 11039 tablet_bootstrap.cc:492] T 4c2a729a802e4abe9ab4f989a40198be P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:15.429498 11039 ts_tablet_manager.cc:1397] T 4c2a729a802e4abe9ab4f989a40198be P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.056s user 0.005s sys 0.004s
I20260924 00:21:15.431491 11039 raft_consensus.cc:348] T 4c2a729a802e4abe9ab4f989a40198be P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:15.432092 11039 raft_consensus.cc:374] T 4c2a729a802e4abe9ab4f989a40198be P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:15.432348 11039 raft_consensus.cc:729] T 4c2a729a802e4abe9ab4f989a40198be P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:15.434090 11039 consensus_queue.cc:260] T 4c2a729a802e4abe9ab4f989a40198be P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:15.444281 11039 ts_tablet_manager.cc:1428] T 4c2a729a802e4abe9ab4f989a40198be P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.015s user 0.004s sys 0.001s
I20260924 00:21:15.446610 11039 tablet_bootstrap.cc:492] T f85ac5b0fd2348d88e1076b781424ab3 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:15.451180 11039 tablet_bootstrap.cc:654] T f85ac5b0fd2348d88e1076b781424ab3 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:15.469341 11869 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b5aa3debe1cd45e19a6bbd0070913ac3)
I20260924 00:21:15.469908 11869 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } }
I20260924 00:21:15.471726 11869 leader_election.cc:288] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1d68651913494ce8afee3c136b2da9cc (127.10.4.126:34479), b5aa3debe1cd45e19a6bbd0070913ac3 (127.10.4.125:34487)
I20260924 00:21:15.473053 10309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" candidate_term: 3 candidate_status { last_received { term: 2 index: 322 } } ignore_live_leader: false dest_uuid: "1d68651913494ce8afee3c136b2da9cc" is_pre_election: true
W20260924 00:21:15.484128 10733 consensus_peers.cc:487] T 675b9e33edcf4d4ea7fa428b476e3945 P 48fbd603abe146c585d15196e1b5be08 -> Peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Couldn't send request to peer b10a9082a37b46f09d581944914c1670. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 675b9e33edcf4d4ea7fa428b476e3945. This is attempt 1: this message will repeat every 5th retry.
I20260924 00:21:15.521335 11719 ts_tablet_manager.cc:1918] T cf36d53828eb421d87b52b3e4cb1d748 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:15.521780 11719 log.cc:1196] T cf36d53828eb421d87b52b3e4cb1d748 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/cf36d53828eb421d87b52b3e4cb1d748
I20260924 00:21:15.522500 11719 ts_tablet_manager.cc:1939] T cf36d53828eb421d87b52b3e4cb1d748 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:15.524490 11719 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 324) took 1768 ms. Trace:
I20260924 00:21:15.538754 11040 tablet_bootstrap.cc:654] T a62b85ebb2b84ae19cad3e3e2dd07a59 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:15.554772 11875 raft_consensus.cc:482] T 9319f9caadb04067a2e9700b5019ed37 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:15.555305 11875 raft_consensus.cc:504] T 9319f9caadb04067a2e9700b5019ed37 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:15.559351 11875 leader_election.cc:288] T 9319f9caadb04067a2e9700b5019ed37 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:15.565299 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9319f9caadb04067a2e9700b5019ed37" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:15.566653 10878 leader_election.cc:341] T 9319f9caadb04067a2e9700b5019ed37 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:15.558898 10653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9319f9caadb04067a2e9700b5019ed37" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:15.573218 10653 raft_consensus.cc:2454] T 9319f9caadb04067a2e9700b5019ed37 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
I20260924 00:21:15.574388 10876 leader_election.cc:302] T 9319f9caadb04067a2e9700b5019ed37 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08
I20260924 00:21:15.561198 11041 tablet_bootstrap.cc:492] T d4e120acff3e4c30aed813b0b24fbfee P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:15.575510 11041 ts_tablet_manager.cc:1397] T d4e120acff3e4c30aed813b0b24fbfee P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.287s user 0.049s sys 0.054s
I20260924 00:21:15.575399 11719 rpcz_store.cc:276] 0924 00:21:13.756336 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:13.757493 (+ 1157us) service_pool.cc:227] Handling call
0924 00:21:14.888510 (+1131017us) ts_tablet_manager.cc:1145] Deleting tablet cf36d53828eb421d87b52b3e4cb1d748
0924 00:21:14.919974 (+ 31464us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:15.159037 (+239063us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:15.520883 (+361846us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:15.524434 (+ 3551us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":7670,"tablet-delete.queue_time_us":1090718}
I20260924 00:21:15.577109 11875 raft_consensus.cc:2781] T 9319f9caadb04067a2e9700b5019ed37 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:15.577487 11875 raft_consensus.cc:482] T 9319f9caadb04067a2e9700b5019ed37 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:15.577929 11875 raft_consensus.cc:3037] T 9319f9caadb04067a2e9700b5019ed37 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:15.580992 11719 tablet_replica.cc:330] T 57ae3d10846a4aaa84eba81b13aafd44 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:15.581765 11719 raft_consensus.cc:2229] T 57ae3d10846a4aaa84eba81b13aafd44 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:15.582238 11719 raft_consensus.cc:2258] T 57ae3d10846a4aaa84eba81b13aafd44 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:15.583855 11719 ts_tablet_manager.cc:1905] T 57ae3d10846a4aaa84eba81b13aafd44 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:15.588332 11854 raft_consensus.cc:482] T a53f06276dbb4d9f97a154a8ceb0278d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:15.588804 11854 raft_consensus.cc:504] T a53f06276dbb4d9f97a154a8ceb0278d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:15.590711 11854 leader_election.cc:288] T a53f06276dbb4d9f97a154a8ceb0278d P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:15.592171 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a53f06276dbb4d9f97a154a8ceb0278d" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:15.593101 11508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1d68651913494ce8afee3c136b2da9cc" candidate_term: 3 candidate_status { last_received { term: 2 index: 453 } } ignore_live_leader: false dest_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" is_pre_election: true
I20260924 00:21:15.593863 11508 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d68651913494ce8afee3c136b2da9cc in term 2.
I20260924 00:21:15.593967 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a53f06276dbb4d9f97a154a8ceb0278d" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:15.595013 10286 leader_election.cc:302] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1d68651913494ce8afee3c136b2da9cc, f8bd0b3184f84f5f8da10c9c21472f92; no voters: b5aa3debe1cd45e19a6bbd0070913ac3
W20260924 00:21:15.595156 10592 leader_election.cc:341] T a53f06276dbb4d9f97a154a8ceb0278d P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
W20260924 00:21:15.596014 10589 leader_election.cc:341] T a53f06276dbb4d9f97a154a8ceb0278d P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:15.596082 11867 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 00:21:15.596383 11867 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 2 FOLLOWER]: Starting leader election (detected failure of leader b5aa3debe1cd45e19a6bbd0070913ac3)
I20260924 00:21:15.596336 10589 leader_election.cc:302] T a53f06276dbb4d9f97a154a8ceb0278d P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:15.596676 11867 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 2 FOLLOWER]: Advancing to term 3
I20260924 00:21:15.597318 11854 raft_consensus.cc:2726] T a53f06276dbb4d9f97a154a8ceb0278d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:15.598088 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet cf36d53828eb421d87b52b3e4cb1d748 (table default.Table_0cdc660142f44fb5b77963f9432b4182 [id=15ae26a9e629499e9c632282502ed104]) successfully deleted
I20260924 00:21:15.601881 10384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" candidate_term: 3 candidate_status { last_received { term: 2 index: 322 } } ignore_live_leader: false dest_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" is_pre_election: true
I20260924 00:21:15.603818 11412 leader_election.cc:302] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f8bd0b3184f84f5f8da10c9c21472f92; no voters: 1d68651913494ce8afee3c136b2da9cc, b5aa3debe1cd45e19a6bbd0070913ac3
I20260924 00:21:15.609531 10908 tablet_service.cc:1514] Processing DeleteTablet for tablet 5107b763acf04df88553ba0af2757570 with delete_type TABLET_DATA_DELETED (replaced by tablet cf0f0f016d074b1fb78d9b427c67fe86) from {username='slave'} at 127.0.0.1:35358
W20260924 00:21:15.610912 10358 catalog_manager.cc:4808] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): delete failed for tablet 5107b763acf04df88553ba0af2757570 because the tablet was not found. No further retry: Not found: Tablet not found: 5107b763acf04df88553ba0af2757570
I20260924 00:21:15.617252 11869 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 00:21:15.617461 10908 tablet_service.cc:1514] Processing DeleteTablet for tablet 9319f9caadb04067a2e9700b5019ed37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:04 UTC) from {username='slave'} at 127.0.0.1:35358
I20260924 00:21:15.618057 11717 tablet_replica.cc:330] T 9319f9caadb04067a2e9700b5019ed37 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:15.628108 11039 tablet_bootstrap.cc:492] T f85ac5b0fd2348d88e1076b781424ab3 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:15.628515 11039 ts_tablet_manager.cc:1397] T f85ac5b0fd2348d88e1076b781424ab3 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.182s user 0.068s sys 0.041s
I20260924 00:21:15.630784 11039 raft_consensus.cc:348] T f85ac5b0fd2348d88e1076b781424ab3 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:15.631080 11875 raft_consensus.cc:504] T 9319f9caadb04067a2e9700b5019ed37 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:15.631465 11039 raft_consensus.cc:374] T f85ac5b0fd2348d88e1076b781424ab3 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:15.631834 11039 raft_consensus.cc:729] T f85ac5b0fd2348d88e1076b781424ab3 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:15.631871 11717 raft_consensus.cc:2229] T 9319f9caadb04067a2e9700b5019ed37 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:15.633555 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9319f9caadb04067a2e9700b5019ed37" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:21:15.633787 11717 raft_consensus.cc:2258] T 9319f9caadb04067a2e9700b5019ed37 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:15.634538 11039 consensus_queue.cc:260] T f85ac5b0fd2348d88e1076b781424ab3 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
W20260924 00:21:15.634951 10878 leader_election.cc:341] T 9319f9caadb04067a2e9700b5019ed37 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:15.636394 11039 ts_tablet_manager.cc:1428] T f85ac5b0fd2348d88e1076b781424ab3 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.008s user 0.001s sys 0.004s
I20260924 00:21:15.588222 11041 raft_consensus.cc:348] T d4e120acff3e4c30aed813b0b24fbfee P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:15.649152 11041 raft_consensus.cc:374] T d4e120acff3e4c30aed813b0b24fbfee P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:15.649437 11041 raft_consensus.cc:729] T d4e120acff3e4c30aed813b0b24fbfee P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:15.649991 11041 consensus_queue.cc:260] T d4e120acff3e4c30aed813b0b24fbfee P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:15.657550 11039 tablet_bootstrap.cc:492] T 717759eb61674c89b05b8d2b0dc34979 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:15.662307 11039 tablet_bootstrap.cc:654] T 717759eb61674c89b05b8d2b0dc34979 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:15.671427 11716 ts_tablet_manager.cc:1918] T cb51d5c6460049108cd7e1974821e578 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:15.673094 11808 raft_consensus.cc:482] T 98926dd3850d40938b67779d5fd9a1db P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:15.673530 11808 raft_consensus.cc:504] T 98926dd3850d40938b67779d5fd9a1db P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:15.675292 11808 leader_election.cc:288] T 98926dd3850d40938b67779d5fd9a1db P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:15.675992 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98926dd3850d40938b67779d5fd9a1db" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:15.677358 10878 leader_election.cc:341] T 98926dd3850d40938b67779d5fd9a1db P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:15.677789 11041 ts_tablet_manager.cc:1428] T d4e120acff3e4c30aed813b0b24fbfee P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.102s user 0.006s sys 0.000s
I20260924 00:21:15.678138 10653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98926dd3850d40938b67779d5fd9a1db" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:15.678653 10653 raft_consensus.cc:2454] T 98926dd3850d40938b67779d5fd9a1db P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
I20260924 00:21:15.679545 11716 log.cc:1196] T cb51d5c6460049108cd7e1974821e578 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/cb51d5c6460049108cd7e1974821e578
I20260924 00:21:15.679850 10876 leader_election.cc:302] T 98926dd3850d40938b67779d5fd9a1db P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08
I20260924 00:21:15.680271 11716 ts_tablet_manager.cc:1939] T cb51d5c6460049108cd7e1974821e578 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:15.680797 11808 raft_consensus.cc:2781] T 98926dd3850d40938b67779d5fd9a1db P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:15.680972 11040 tablet_bootstrap.cc:492] T a62b85ebb2b84ae19cad3e3e2dd07a59 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:15.681159 11808 raft_consensus.cc:482] T 98926dd3850d40938b67779d5fd9a1db P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:15.681336 11040 ts_tablet_manager.cc:1397] T a62b85ebb2b84ae19cad3e3e2dd07a59 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.279s user 0.006s sys 0.003s
I20260924 00:21:15.681425 11808 raft_consensus.cc:3037] T 98926dd3850d40938b67779d5fd9a1db P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:15.682231 11716 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35996 (request call id 328) took 1778 ms. Trace:
I20260924 00:21:15.682515 11716 rpcz_store.cc:276] 0924 00:21:13.904251 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:13.909104 (+ 4853us) service_pool.cc:227] Handling call
0924 00:21:15.203582 (+1294478us) ts_tablet_manager.cc:1145] Deleting tablet cb51d5c6460049108cd7e1974821e578
0924 00:21:15.203632 (+ 50us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:15.569780 (+366148us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:15.671213 (+101433us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:15.682177 (+ 10964us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1293924}
I20260924 00:21:15.683385 11040 raft_consensus.cc:348] T a62b85ebb2b84ae19cad3e3e2dd07a59 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:15.683817 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet cb51d5c6460049108cd7e1974821e578 (table default.Table_9172ff8a00a54ed581e96de0f0b44c3c [id=ed975cbe1bb04e3194882bed0f4ec0b1]) successfully deleted
I20260924 00:21:15.683991 11040 raft_consensus.cc:374] T a62b85ebb2b84ae19cad3e3e2dd07a59 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:15.684199 11040 raft_consensus.cc:729] T a62b85ebb2b84ae19cad3e3e2dd07a59 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:15.684373 11875 leader_election.cc:288] T 9319f9caadb04067a2e9700b5019ed37 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:15.685668 11041 tablet_bootstrap.cc:492] T 1d5a76bc420b41098d4ebd3dc59aa1fc P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:15.685622 10653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9319f9caadb04067a2e9700b5019ed37" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:21:15.686131 11717 ts_tablet_manager.cc:1905] T 9319f9caadb04067a2e9700b5019ed37 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:15.686156 10653 raft_consensus.cc:3037] T 9319f9caadb04067a2e9700b5019ed37 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:15.691426 10624 tablet_service.cc:1514] Processing DeleteTablet for tablet d4e120acff3e4c30aed813b0b24fbfee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:03 UTC) from {username='slave'} at 127.0.0.1:46896
I20260924 00:21:15.695348 10653 raft_consensus.cc:2454] T 9319f9caadb04067a2e9700b5019ed37 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 1.
I20260924 00:21:15.701993 10876 leader_election.cc:302] T 9319f9caadb04067a2e9700b5019ed37 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08
I20260924 00:21:15.702452 11041 tablet_bootstrap.cc:654] T 1d5a76bc420b41098d4ebd3dc59aa1fc P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
W20260924 00:21:15.702556 10876 raft_consensus.cc:2670] T 9319f9caadb04067a2e9700b5019ed37 P c8626e300d0644b0908797ad61bd7cef: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260924 00:21:15.709056 11040 consensus_queue.cc:260] T a62b85ebb2b84ae19cad3e3e2dd07a59 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:15.675889 11867 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } }
I20260924 00:21:15.713881 11717 ts_tablet_manager.cc:1918] T 9319f9caadb04067a2e9700b5019ed37 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:15.714434 11040 ts_tablet_manager.cc:1428] T a62b85ebb2b84ae19cad3e3e2dd07a59 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.033s user 0.005s sys 0.000s
I20260924 00:21:15.719692 11040 tablet_bootstrap.cc:492] T cf36d53828eb421d87b52b3e4cb1d748 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:15.730140 11867 leader_election.cc:288] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [CANDIDATE]: Term 3 election: Requested vote from peers b5aa3debe1cd45e19a6bbd0070913ac3 (127.10.4.125:34487), f8bd0b3184f84f5f8da10c9c21472f92 (127.10.4.124:35345)
I20260924 00:21:15.715871 11039 tablet_bootstrap.cc:492] T 717759eb61674c89b05b8d2b0dc34979 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:15.733238 11039 ts_tablet_manager.cc:1397] T 717759eb61674c89b05b8d2b0dc34979 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.076s user 0.007s sys 0.001s
I20260924 00:21:15.733978 11508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1d68651913494ce8afee3c136b2da9cc" candidate_term: 3 candidate_status { last_received { term: 2 index: 453 } } ignore_live_leader: false dest_uuid: "f8bd0b3184f84f5f8da10c9c21472f92"
I20260924 00:21:15.734540 11508 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 2 FOLLOWER]: Advancing to term 3
I20260924 00:21:15.735343 11039 raft_consensus.cc:348] T 717759eb61674c89b05b8d2b0dc34979 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:15.735924 11039 raft_consensus.cc:374] T 717759eb61674c89b05b8d2b0dc34979 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:15.736161 11039 raft_consensus.cc:729] T 717759eb61674c89b05b8d2b0dc34979 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:15.737202 11771 raft_consensus.cc:482] T 717759eb61674c89b05b8d2b0dc34979 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:15.736852 11039 consensus_queue.cc:260] T 717759eb61674c89b05b8d2b0dc34979 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:15.737843 11717 log.cc:1196] T 9319f9caadb04067a2e9700b5019ed37 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/9319f9caadb04067a2e9700b5019ed37
I20260924 00:21:15.737618 11771 raft_consensus.cc:504] T 717759eb61674c89b05b8d2b0dc34979 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:15.738550 11717 ts_tablet_manager.cc:1939] T 9319f9caadb04067a2e9700b5019ed37 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:15.739425 11771 leader_election.cc:288] T 717759eb61674c89b05b8d2b0dc34979 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:15.741725 10384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1d68651913494ce8afee3c136b2da9cc" candidate_term: 3 candidate_status { last_received { term: 2 index: 453 } } ignore_live_leader: false dest_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3"
W20260924 00:21:15.744503 10733 leader_election.cc:341] T 717759eb61674c89b05b8d2b0dc34979 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: 717759eb61674c89b05b8d2b0dc34979
I20260924 00:21:15.746490 11039 ts_tablet_manager.cc:1428] T 717759eb61674c89b05b8d2b0dc34979 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.013s user 0.000s sys 0.004s
I20260924 00:21:15.747318 11039 tablet_bootstrap.cc:492] T c7cb7c6892404c7fa593cdaaac67f800 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:15.750411 11875 raft_consensus.cc:482] T 4c2a729a802e4abe9ab4f989a40198be P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:15.750859 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "717759eb61674c89b05b8d2b0dc34979" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:15.751003 11808 raft_consensus.cc:504] T 98926dd3850d40938b67779d5fd9a1db P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:15.751380 10942 raft_consensus.cc:2454] T 717759eb61674c89b05b8d2b0dc34979 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 0.
I20260924 00:21:15.752243 11799 raft_consensus.cc:482] T 57ae3d10846a4aaa84eba81b13aafd44 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:15.752615 11799 raft_consensus.cc:504] T 57ae3d10846a4aaa84eba81b13aafd44 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:15.752871 10732 leader_election.cc:302] T 717759eb61674c89b05b8d2b0dc34979 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters: b10a9082a37b46f09d581944914c1670
I20260924 00:21:15.753894 11771 raft_consensus.cc:2781] T 717759eb61674c89b05b8d2b0dc34979 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:15.754185 11771 raft_consensus.cc:482] T 717759eb61674c89b05b8d2b0dc34979 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:15.754335 11799 leader_election.cc:288] T 57ae3d10846a4aaa84eba81b13aafd44 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:15.754485 11771 raft_consensus.cc:3037] T 717759eb61674c89b05b8d2b0dc34979 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:15.750887 11875 raft_consensus.cc:504] T 4c2a729a802e4abe9ab4f989a40198be P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:15.757884 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57ae3d10846a4aaa84eba81b13aafd44" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:15.758812 10652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c2a729a802e4abe9ab4f989a40198be" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:15.759479 10652 raft_consensus.cc:2454] T 4c2a729a802e4abe9ab4f989a40198be P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
W20260924 00:21:15.760908 10876 leader_election.cc:341] T 57ae3d10846a4aaa84eba81b13aafd44 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Illegal state: Tablet not RUNNING: STOPPED
I20260924 00:21:15.761510 10876 leader_election.cc:302] T 4c2a729a802e4abe9ab4f989a40198be P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:15.762220 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 9319f9caadb04067a2e9700b5019ed37 (table default.Table_4c54324167a8434a90e8dcb1354111f9 [id=9da4000ba42645d69f7f3c5fc3a941c5]) successfully deleted
I20260924 00:21:15.762521 11799 raft_consensus.cc:2781] T 4c2a729a802e4abe9ab4f989a40198be P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:15.762842 11799 raft_consensus.cc:482] T 4c2a729a802e4abe9ab4f989a40198be P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:15.763139 11799 raft_consensus.cc:3037] T 4c2a729a802e4abe9ab4f989a40198be P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:15.765868 10774 tablet_service.cc:1514] Processing DeleteTablet for tablet a62b85ebb2b84ae19cad3e3e2dd07a59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:09 UTC) from {username='slave'} at 127.0.0.1:35996
I20260924 00:21:15.769191 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57ae3d10846a4aaa84eba81b13aafd44" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:15.770154 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c2a729a802e4abe9ab4f989a40198be" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:15.771202 11716 tablet_replica.cc:330] T b36cbb6520de4df58b88cfa53c20ee7a P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
W20260924 00:21:15.771128 10878 leader_election.cc:341] T 57ae3d10846a4aaa84eba81b13aafd44 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:15.771477 10878 leader_election.cc:302] T 57ae3d10846a4aaa84eba81b13aafd44 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670
I20260924 00:21:15.771852 11716 raft_consensus.cc:2229] T b36cbb6520de4df58b88cfa53c20ee7a P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:15.772233 11716 raft_consensus.cc:2258] T b36cbb6520de4df58b88cfa53c20ee7a P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260924 00:21:15.772431 10878 leader_election.cc:341] T 4c2a729a802e4abe9ab4f989a40198be P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:15.772922 11807 raft_consensus.cc:2726] T 57ae3d10846a4aaa84eba81b13aafd44 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:15.774977 11799 raft_consensus.cc:504] T 4c2a729a802e4abe9ab4f989a40198be P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:15.777513 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c2a729a802e4abe9ab4f989a40198be" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:21:15.777776 10652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c2a729a802e4abe9ab4f989a40198be" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:21:15.778302 10652 raft_consensus.cc:3037] T 4c2a729a802e4abe9ab4f989a40198be P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
W20260924 00:21:15.778614 10878 leader_election.cc:341] T 4c2a729a802e4abe9ab4f989a40198be P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:15.782370 11039 tablet_bootstrap.cc:654] T c7cb7c6892404c7fa593cdaaac67f800 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:15.784900 10652 raft_consensus.cc:2454] T 4c2a729a802e4abe9ab4f989a40198be P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 1.
I20260924 00:21:15.788013 11508 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d68651913494ce8afee3c136b2da9cc in term 3.
I20260924 00:21:15.789098 10286 leader_election.cc:302] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1d68651913494ce8afee3c136b2da9cc, f8bd0b3184f84f5f8da10c9c21472f92; no voters: b5aa3debe1cd45e19a6bbd0070913ac3
I20260924 00:21:15.790060 11867 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 3 FOLLOWER]: Leader election won for term 3
I20260924 00:21:15.791085 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98926dd3850d40938b67779d5fd9a1db" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:21:15.801570 11716 ts_tablet_manager.cc:1905] T b36cbb6520de4df58b88cfa53c20ee7a P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:15.809422 11771 raft_consensus.cc:504] T 717759eb61674c89b05b8d2b0dc34979 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:15.785386 11875 leader_election.cc:288] T 4c2a729a802e4abe9ab4f989a40198be P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:15.786545 10876 leader_election.cc:302] T 4c2a729a802e4abe9ab4f989a40198be P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08
I20260924 00:21:15.789216 11808 leader_election.cc:288] T 98926dd3850d40938b67779d5fd9a1db P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:15.793259 11799 leader_election.cc:288] T 4c2a729a802e4abe9ab4f989a40198be P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
W20260924 00:21:15.797533 10878 leader_election.cc:341] T 98926dd3850d40938b67779d5fd9a1db P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:15.812675 11808 raft_consensus.cc:2781] T 4c2a729a802e4abe9ab4f989a40198be P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:15.814826 11808 raft_consensus.cc:686] T 4c2a729a802e4abe9ab4f989a40198be P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Becoming Leader. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Running, Role: LEADER
I20260924 00:21:15.815163 10653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98926dd3850d40938b67779d5fd9a1db" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:21:15.816521 11808 consensus_queue.cc:237] T 4c2a729a802e4abe9ab4f989a40198be P c8626e300d0644b0908797ad61bd7cef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:15.819016 10653 raft_consensus.cc:3037] T 98926dd3850d40938b67779d5fd9a1db P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:15.821022 11867 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 3 LEADER]: Becoming Leader. State: Replica: 1d68651913494ce8afee3c136b2da9cc, State: Running, Role: LEADER
I20260924 00:21:15.821998 11867 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 453, Committed index: 453, Last appended: 2.453, Last appended by leader: 453, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } }
I20260924 00:21:15.833575 11041 tablet_bootstrap.cc:492] T 1d5a76bc420b41098d4ebd3dc59aa1fc P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:15.834067 11041 ts_tablet_manager.cc:1397] T 1d5a76bc420b41098d4ebd3dc59aa1fc P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.149s user 0.008s sys 0.001s
I20260924 00:21:15.836304 11041 raft_consensus.cc:348] T 1d5a76bc420b41098d4ebd3dc59aa1fc P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:15.839594 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "717759eb61674c89b05b8d2b0dc34979" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef"
I20260924 00:21:15.840111 10942 raft_consensus.cc:3037] T 717759eb61674c89b05b8d2b0dc34979 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:15.847534 10942 raft_consensus.cc:2454] T 717759eb61674c89b05b8d2b0dc34979 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48fbd603abe146c585d15196e1b5be08 in term 1.
I20260924 00:21:15.848645 10732 leader_election.cc:302] T 717759eb61674c89b05b8d2b0dc34979 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:15.854988 10309 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [term 3 LEADER]: Rejecting Update request from peer b5aa3debe1cd45e19a6bbd0070913ac3 for earlier term 2. Current term is 3. Ops: []
I20260924 00:21:15.856937 11868 consensus_queue.cc:1046] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 }, Status: INVALID_TERM, Last received: 2.453, Next index: 454, Last known committed idx: 453, Time since last communication: 0.001s
I20260924 00:21:15.859023 11039 tablet_bootstrap.cc:492] T c7cb7c6892404c7fa593cdaaac67f800 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:15.859439 11039 ts_tablet_manager.cc:1397] T c7cb7c6892404c7fa593cdaaac67f800 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.112s user 0.005s sys 0.003s
I20260924 00:21:15.869182 11868 consensus_queue.cc:1225] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [LEADER]: Peer 1d68651913494ce8afee3c136b2da9cc log is divergent from this leader: its last log entry 3.454 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 453
I20260924 00:21:15.873113 11886 raft_consensus.cc:3032] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 00:21:15.873754 11886 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b5aa3debe1cd45e19a6bbd0070913ac3, State: Running, Role: LEADER
I20260924 00:21:15.876678 11716 ts_tablet_manager.cc:1918] T b36cbb6520de4df58b88cfa53c20ee7a P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:15.879439 11886 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 453, Committed index: 453, Last appended: 2.453, Last appended by leader: 453, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } }
I20260924 00:21:15.878823 11881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d68651913494ce8afee3c136b2da9cc. Latest consensus state: current_term: 3 leader_uuid: "1d68651913494ce8afee3c136b2da9cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } } }
I20260924 00:21:15.881064 11041 raft_consensus.cc:374] T 1d5a76bc420b41098d4ebd3dc59aa1fc P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:15.881345 11041 raft_consensus.cc:729] T 1d5a76bc420b41098d4ebd3dc59aa1fc P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:15.881906 11041 consensus_queue.cc:260] T 1d5a76bc420b41098d4ebd3dc59aa1fc P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:15.882968 10653 raft_consensus.cc:2454] T 98926dd3850d40938b67779d5fd9a1db P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 1.
I20260924 00:21:15.883739 11041 ts_tablet_manager.cc:1428] T 1d5a76bc420b41098d4ebd3dc59aa1fc P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.049s user 0.002s sys 0.001s
I20260924 00:21:15.838243 11719 ts_tablet_manager.cc:1918] T 57ae3d10846a4aaa84eba81b13aafd44 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:15.883989 10876 leader_election.cc:302] T 98926dd3850d40938b67779d5fd9a1db P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08
I20260924 00:21:15.884297 11719 log.cc:1196] T 57ae3d10846a4aaa84eba81b13aafd44 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/57ae3d10846a4aaa84eba81b13aafd44
I20260924 00:21:15.884862 11719 ts_tablet_manager.cc:1939] T 57ae3d10846a4aaa84eba81b13aafd44 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:15.885116 11808 raft_consensus.cc:2781] T 98926dd3850d40938b67779d5fd9a1db P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:15.885620 11808 raft_consensus.cc:686] T 98926dd3850d40938b67779d5fd9a1db P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Becoming Leader. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Running, Role: LEADER
I20260924 00:21:15.886432 11808 consensus_queue.cc:237] T 98926dd3850d40938b67779d5fd9a1db P c8626e300d0644b0908797ad61bd7cef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:15.897743 11039 raft_consensus.cc:348] T c7cb7c6892404c7fa593cdaaac67f800 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:15.900151 11039 raft_consensus.cc:374] T c7cb7c6892404c7fa593cdaaac67f800 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:15.900411 11039 raft_consensus.cc:729] T c7cb7c6892404c7fa593cdaaac67f800 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:15.901561 11881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [sys.catalog]: This master's current role is: LEADER
I20260924 00:21:15.902788 11886 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [term 2 FOLLOWER]: Advancing to term 3
I20260924 00:21:15.906173 11719 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 327) took 2111 ms. Trace:
I20260924 00:21:15.906471 11719 rpcz_store.cc:276] 0924 00:21:13.794934 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:13.824598 (+ 29664us) service_pool.cc:227] Handling call
0924 00:21:15.576817 (+1752219us) ts_tablet_manager.cc:1145] Deleting tablet 57ae3d10846a4aaa84eba81b13aafd44
0924 00:21:15.576861 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:15.681459 (+104598us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:15.838033 (+156574us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:15.906103 (+ 68070us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1719420}
I20260924 00:21:15.907500 11719 tablet_replica.cc:330] T 9319f9caadb04067a2e9700b5019ed37 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:15.908130 11719 raft_consensus.cc:2229] T 9319f9caadb04067a2e9700b5019ed37 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:15.908628 11719 raft_consensus.cc:2258] T 9319f9caadb04067a2e9700b5019ed37 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:15.908566 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 57ae3d10846a4aaa84eba81b13aafd44 (table default.Table_29e523da31634984b775f31084e82c99 [id=6d2a4bce1272466c8a00217643c0c9f5]) successfully deleted
I20260924 00:21:15.910147 11719 ts_tablet_manager.cc:1905] T 9319f9caadb04067a2e9700b5019ed37 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:15.918056 11888 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:21:15.923899 11039 consensus_queue.cc:260] T c7cb7c6892404c7fa593cdaaac67f800 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:15.926407 11039 ts_tablet_manager.cc:1428] T c7cb7c6892404c7fa593cdaaac67f800 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.067s user 0.006s sys 0.001s
I20260924 00:21:15.927341 11039 tablet_bootstrap.cc:492] T 2e839fd66fed4472a0892baa9aa85c54 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:15.932806 11041 tablet_bootstrap.cc:492] T c7cb7c6892404c7fa593cdaaac67f800 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:15.959817 11041 tablet_bootstrap.cc:654] T c7cb7c6892404c7fa593cdaaac67f800 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:15.968214 11799 raft_consensus.cc:482] T c7cb7c6892404c7fa593cdaaac67f800 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:15.968633 11799 raft_consensus.cc:504] T c7cb7c6892404c7fa593cdaaac67f800 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:15.970371 11799 leader_election.cc:288] T c7cb7c6892404c7fa593cdaaac67f800 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:15.973135 10653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7cb7c6892404c7fa593cdaaac67f800" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
W20260924 00:21:15.974303 10876 leader_election.cc:341] T c7cb7c6892404c7fa593cdaaac67f800 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:15.877174 11885 raft_consensus.cc:2781] T 717759eb61674c89b05b8d2b0dc34979 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:15.977119 11716 log.cc:1196] T b36cbb6520de4df58b88cfa53c20ee7a P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/b36cbb6520de4df58b88cfa53c20ee7a
I20260924 00:21:15.877594 11771 leader_election.cc:288] T 717759eb61674c89b05b8d2b0dc34979 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Requested vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
W20260924 00:21:15.878319 10733 leader_election.cc:341] T 717759eb61674c89b05b8d2b0dc34979 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: 717759eb61674c89b05b8d2b0dc34979
I20260924 00:21:15.977851 11716 ts_tablet_manager.cc:1939] T b36cbb6520de4df58b88cfa53c20ee7a P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
W20260924 00:21:15.978168 10733 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:33461, user_credentials={real_user=slave}} blocked reactor thread for 99886.8us
I20260924 00:21:15.883749 11040 tablet_bootstrap.cc:654] T cf36d53828eb421d87b52b3e4cb1d748 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:15.979730 11716 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35996 (request call id 329) took 2069 ms. Trace:
I20260924 00:21:15.980003 11716 rpcz_store.cc:276] 0924 00:21:13.910092 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:13.934439 (+ 24347us) service_pool.cc:227] Handling call
0924 00:21:15.716199 (+1781760us) ts_tablet_manager.cc:1145] Deleting tablet b36cbb6520de4df58b88cfa53c20ee7a
0924 00:21:15.768995 (+ 52796us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:15.870018 (+101023us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:15.876484 (+ 6466us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:15.979674 (+103190us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1781237}
I20260924 00:21:15.981006 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet b36cbb6520de4df58b88cfa53c20ee7a (table default.Table_dae0ffc8534047588f6f7cb13a610af5 [id=138be013fbcb4f01a6358d062200fe8f]) successfully deleted
I20260924 00:21:15.971760 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7cb7c6892404c7fa593cdaaac67f800" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:15.983119 10878 leader_election.cc:341] T c7cb7c6892404c7fa593cdaaac67f800 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:15.983454 10878 leader_election.cc:302] T c7cb7c6892404c7fa593cdaaac67f800 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670
I20260924 00:21:15.984083 11799 raft_consensus.cc:2726] T c7cb7c6892404c7fa593cdaaac67f800 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:15.985392 11771 raft_consensus.cc:482] T a62b85ebb2b84ae19cad3e3e2dd07a59 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:15.985809 11771 raft_consensus.cc:504] T a62b85ebb2b84ae19cad3e3e2dd07a59 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:15.987489 11771 leader_election.cc:288] T a62b85ebb2b84ae19cad3e3e2dd07a59 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
W20260924 00:21:15.989634 10732 leader_election.cc:341] T a62b85ebb2b84ae19cad3e3e2dd07a59 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Not found: Tablet not found: a62b85ebb2b84ae19cad3e3e2dd07a59
W20260924 00:21:15.993353 10733 leader_election.cc:341] T a62b85ebb2b84ae19cad3e3e2dd07a59 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: a62b85ebb2b84ae19cad3e3e2dd07a59
I20260924 00:21:15.993719 10733 leader_election.cc:302] T a62b85ebb2b84ae19cad3e3e2dd07a59 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08; no voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:15.996078 11771 raft_consensus.cc:2726] T a62b85ebb2b84ae19cad3e3e2dd07a59 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:15.998464 11039 tablet_bootstrap.cc:654] T 2e839fd66fed4472a0892baa9aa85c54 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:16.012544 11040 tablet_bootstrap.cc:492] T cf36d53828eb421d87b52b3e4cb1d748 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:16.013034 11040 ts_tablet_manager.cc:1397] T cf36d53828eb421d87b52b3e4cb1d748 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.293s user 0.015s sys 0.002s
I20260924 00:21:16.015708 11040 raft_consensus.cc:348] T cf36d53828eb421d87b52b3e4cb1d748 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.016340 11040 raft_consensus.cc:374] T cf36d53828eb421d87b52b3e4cb1d748 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:16.016573 11040 raft_consensus.cc:729] T cf36d53828eb421d87b52b3e4cb1d748 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:16.020576 11885 raft_consensus.cc:686] T 717759eb61674c89b05b8d2b0dc34979 P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: Becoming Leader. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Running, Role: LEADER
I20260924 00:21:16.021566 11885 consensus_queue.cc:237] T 717759eb61674c89b05b8d2b0dc34979 P 48fbd603abe146c585d15196e1b5be08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.020622 11716 tablet_replica.cc:330] T 59a0a950a8144e99b2ed47be9a567d9a P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:16.023830 11716 raft_consensus.cc:2229] T 59a0a950a8144e99b2ed47be9a567d9a P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:16.024283 11716 raft_consensus.cc:2258] T 59a0a950a8144e99b2ed47be9a567d9a P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:16.026611 11039 tablet_bootstrap.cc:492] T 2e839fd66fed4472a0892baa9aa85c54 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:16.027062 11039 ts_tablet_manager.cc:1397] T 2e839fd66fed4472a0892baa9aa85c54 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.100s user 0.008s sys 0.001s
I20260924 00:21:16.025907 11716 ts_tablet_manager.cc:1905] T 59a0a950a8144e99b2ed47be9a567d9a P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:16.029150 11039 raft_consensus.cc:348] T 2e839fd66fed4472a0892baa9aa85c54 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.029753 11039 raft_consensus.cc:374] T 2e839fd66fed4472a0892baa9aa85c54 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:16.029986 11039 raft_consensus.cc:729] T 2e839fd66fed4472a0892baa9aa85c54 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:16.030545 11039 consensus_queue.cc:260] T 2e839fd66fed4472a0892baa9aa85c54 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.061172 11040 consensus_queue.cc:260] T cf36d53828eb421d87b52b3e4cb1d748 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.063342 11040 ts_tablet_manager.cc:1428] T cf36d53828eb421d87b52b3e4cb1d748 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.050s user 0.006s sys 0.000s
I20260924 00:21:16.077086 11719 ts_tablet_manager.cc:1918] T 9319f9caadb04067a2e9700b5019ed37 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:16.077471 11719 log.cc:1196] T 9319f9caadb04067a2e9700b5019ed37 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/9319f9caadb04067a2e9700b5019ed37
I20260924 00:21:16.077914 11041 tablet_bootstrap.cc:492] T c7cb7c6892404c7fa593cdaaac67f800 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:16.078205 11719 ts_tablet_manager.cc:1939] T 9319f9caadb04067a2e9700b5019ed37 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:16.078383 11041 ts_tablet_manager.cc:1397] T c7cb7c6892404c7fa593cdaaac67f800 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.146s user 0.008s sys 0.001s
I20260924 00:21:16.080096 11719 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 328) took 2254 ms. Trace:
I20260924 00:21:16.080397 11719 rpcz_store.cc:276] 0924 00:21:13.825930 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:13.826019 (+ 89us) service_pool.cc:227] Handling call
0924 00:21:15.907372 (+2081353us) ts_tablet_manager.cc:1145] Deleting tablet 9319f9caadb04067a2e9700b5019ed37
0924 00:21:15.907406 (+ 34us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:15.974788 (+ 67382us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:16.035477 (+ 60689us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:16.080043 (+ 44566us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2048208}
I20260924 00:21:16.080883 11041 raft_consensus.cc:348] T c7cb7c6892404c7fa593cdaaac67f800 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.081506 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 9319f9caadb04067a2e9700b5019ed37 (table default.Table_4c54324167a8434a90e8dcb1354111f9 [id=9da4000ba42645d69f7f3c5fc3a941c5]) successfully deleted
I20260924 00:21:16.081691 11041 raft_consensus.cc:374] T c7cb7c6892404c7fa593cdaaac67f800 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:16.082051 11041 raft_consensus.cc:729] T c7cb7c6892404c7fa593cdaaac67f800 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:16.083499 11041 consensus_queue.cc:260] T c7cb7c6892404c7fa593cdaaac67f800 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.085716 11041 ts_tablet_manager.cc:1428] T c7cb7c6892404c7fa593cdaaac67f800 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.007s user 0.007s sys 0.001s
I20260924 00:21:16.107206 11039 ts_tablet_manager.cc:1428] T 2e839fd66fed4472a0892baa9aa85c54 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.080s user 0.004s sys 0.000s
I20260924 00:21:16.108209 11039 tablet_bootstrap.cc:492] T 28be3abcaff64a3d97fad866472b41be P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:16.121618 11041 tablet_bootstrap.cc:492] T f85ac5b0fd2348d88e1076b781424ab3 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
W20260924 00:21:16.124347 10359 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 7673597f1afc43198e86929a2939269a (table default.Table_b36ac15778564d608790f586457c6373 [id=93c1734de58245f8b1d5ab88ec9f0c8e], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 7673597f1afc43198e86929a2939269a consensus state has no leader
I20260924 00:21:16.133947 11040 tablet_bootstrap.cc:492] T 4c2a729a802e4abe9ab4f989a40198be P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:16.159122 11041 tablet_bootstrap.cc:654] T f85ac5b0fd2348d88e1076b781424ab3 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:16.161072 11719 tablet_replica.cc:330] T 5107b763acf04df88553ba0af2757570 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:16.161986 11719 raft_consensus.cc:2229] T 5107b763acf04df88553ba0af2757570 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:16.162436 11719 raft_consensus.cc:2258] T 5107b763acf04df88553ba0af2757570 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:16.164453 11719 ts_tablet_manager.cc:1905] T 5107b763acf04df88553ba0af2757570 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:16.170071 11040 tablet_bootstrap.cc:654] T 4c2a729a802e4abe9ab4f989a40198be P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:16.170763 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_a01f8ccbfb7a4cff9315ec54f8f2f50e [id=001b691754074a66b8bbc7fa0f95caff]
I20260924 00:21:16.174494 11041 tablet_bootstrap.cc:492] T f85ac5b0fd2348d88e1076b781424ab3 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:16.174878 11041 ts_tablet_manager.cc:1397] T f85ac5b0fd2348d88e1076b781424ab3 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.054s user 0.007s sys 0.001s
I20260924 00:21:16.176287 11892 raft_consensus.cc:482] T d4e120acff3e4c30aed813b0b24fbfee P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:16.176692 11892 raft_consensus.cc:504] T d4e120acff3e4c30aed813b0b24fbfee P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:16.178449 11892 leader_election.cc:288] T d4e120acff3e4c30aed813b0b24fbfee P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:16.178689 11039 tablet_bootstrap.cc:654] T 28be3abcaff64a3d97fad866472b41be P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:16.180640 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4e120acff3e4c30aed813b0b24fbfee" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:16.181770 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4e120acff3e4c30aed813b0b24fbfee" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:16.183280 10592 leader_election.cc:341] T d4e120acff3e4c30aed813b0b24fbfee P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
W20260924 00:21:16.184348 10589 leader_election.cc:341] T d4e120acff3e4c30aed813b0b24fbfee P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:16.184725 10589 leader_election.cc:302] T d4e120acff3e4c30aed813b0b24fbfee P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b10a9082a37b46f09d581944914c1670; no voters: 48fbd603abe146c585d15196e1b5be08, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:16.180596 11716 ts_tablet_manager.cc:1918] T 59a0a950a8144e99b2ed47be9a567d9a P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:16.185266 11716 log.cc:1196] T 59a0a950a8144e99b2ed47be9a567d9a P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/59a0a950a8144e99b2ed47be9a567d9a
I20260924 00:21:16.185812 11716 ts_tablet_manager.cc:1939] T 59a0a950a8144e99b2ed47be9a567d9a P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:16.185819 11892 raft_consensus.cc:2726] T d4e120acff3e4c30aed813b0b24fbfee P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:16.187479 11716 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35996 (request call id 330) took 2252 ms. Trace:
I20260924 00:21:16.187453 11041 raft_consensus.cc:348] T f85ac5b0fd2348d88e1076b781424ab3 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.195930 11041 raft_consensus.cc:374] T f85ac5b0fd2348d88e1076b781424ab3 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:16.196300 11041 raft_consensus.cc:729] T f85ac5b0fd2348d88e1076b781424ab3 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:16.197530 11041 consensus_queue.cc:260] T f85ac5b0fd2348d88e1076b781424ab3 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.187757 11716 rpcz_store.cc:276] 0924 00:21:13.934870 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:13.934984 (+ 114us) service_pool.cc:227] Handling call
0924 00:21:16.020471 (+2085487us) ts_tablet_manager.cc:1145] Deleting tablet 59a0a950a8144e99b2ed47be9a567d9a
0924 00:21:16.020515 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:16.151907 (+131392us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:16.180390 (+ 28483us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:16.187427 (+ 7037us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2085001}
I20260924 00:21:16.199380 11716 tablet_replica.cc:330] T 3b8d74761b0b4fd0a22021d3870ec05c P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:16.207558 11716 raft_consensus.cc:2229] T 3b8d74761b0b4fd0a22021d3870ec05c P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:16.208022 11716 raft_consensus.cc:2258] T 3b8d74761b0b4fd0a22021d3870ec05c P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:16.209826 11716 ts_tablet_manager.cc:1905] T 3b8d74761b0b4fd0a22021d3870ec05c P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:16.210631 11040 tablet_bootstrap.cc:492] T 4c2a729a802e4abe9ab4f989a40198be P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:16.211004 11040 ts_tablet_manager.cc:1397] T 4c2a729a802e4abe9ab4f989a40198be P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.077s user 0.008s sys 0.000s
I20260924 00:21:16.212893 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 59a0a950a8144e99b2ed47be9a567d9a (table default.Table_faaa8b08d77d4384a41b2008c9cc98ea [id=3fce8b5a6e82492b8a2355183e566142]) successfully deleted
I20260924 00:21:16.225636 11041 ts_tablet_manager.cc:1428] T f85ac5b0fd2348d88e1076b781424ab3 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.050s user 0.003s sys 0.002s
I20260924 00:21:16.226259 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_b4a6ce0bb6394e0aadb54d15db1f4b80 [id=02df80a9de5947dbbf8484189637255b]
I20260924 00:21:16.229167 11040 raft_consensus.cc:348] T 4c2a729a802e4abe9ab4f989a40198be P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.229800 11040 raft_consensus.cc:374] T 4c2a729a802e4abe9ab4f989a40198be P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:16.230026 11040 raft_consensus.cc:729] T 4c2a729a802e4abe9ab4f989a40198be P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:16.230749 11040 consensus_queue.cc:260] T 4c2a729a802e4abe9ab4f989a40198be P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.232992 11040 ts_tablet_manager.cc:1428] T 4c2a729a802e4abe9ab4f989a40198be P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.022s user 0.007s sys 0.000s
I20260924 00:21:16.246248 11040 tablet_bootstrap.cc:492] T 57ae3d10846a4aaa84eba81b13aafd44 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:16.251852 11040 tablet_bootstrap.cc:654] T 57ae3d10846a4aaa84eba81b13aafd44 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:16.266045 11039 tablet_bootstrap.cc:492] T 28be3abcaff64a3d97fad866472b41be P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:16.266435 11039 ts_tablet_manager.cc:1397] T 28be3abcaff64a3d97fad866472b41be P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.158s user 0.008s sys 0.000s
I20260924 00:21:16.269469 11716 ts_tablet_manager.cc:1918] T 3b8d74761b0b4fd0a22021d3870ec05c P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:16.269824 11716 log.cc:1196] T 3b8d74761b0b4fd0a22021d3870ec05c P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/3b8d74761b0b4fd0a22021d3870ec05c
I20260924 00:21:16.270480 11716 ts_tablet_manager.cc:1939] T 3b8d74761b0b4fd0a22021d3870ec05c P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:16.271685 11041 tablet_bootstrap.cc:492] T 28be3abcaff64a3d97fad866472b41be P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:16.272784 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_4c92eecf56284783b041439a866855b8 [id=0311178133ae4be895516123aa4ea4fd]
I20260924 00:21:16.277994 11041 tablet_bootstrap.cc:654] T 28be3abcaff64a3d97fad866472b41be P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:16.279641 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_b5c9933d8e0c481290863b60a642e8e7 [id=036fc1e2f392468ca0e9ce72610f3620]
I20260924 00:21:16.268464 11039 raft_consensus.cc:348] T 28be3abcaff64a3d97fad866472b41be P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.283252 11039 raft_consensus.cc:374] T 28be3abcaff64a3d97fad866472b41be P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:16.283515 11039 raft_consensus.cc:729] T 28be3abcaff64a3d97fad866472b41be P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:16.284634 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_ed94e40b06bf4340bd3a775e16a6d139 [id=0742097f4ab743959806cffe991bb5e5]
I20260924 00:21:16.285686 11716 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35996 (request call id 331) took 2350 ms. Trace:
I20260924 00:21:16.286087 11040 tablet_bootstrap.cc:492] T 57ae3d10846a4aaa84eba81b13aafd44 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:16.285971 11716 rpcz_store.cc:276] 0924 00:21:13.935178 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:13.935265 (+ 87us) service_pool.cc:227] Handling call
0924 00:21:16.199230 (+2263965us) ts_tablet_manager.cc:1145] Deleting tablet 3b8d74761b0b4fd0a22021d3870ec05c
0924 00:21:16.199281 (+ 51us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:16.219927 (+ 20646us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:16.269277 (+ 49350us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:16.285630 (+ 16353us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2263500}
I20260924 00:21:16.287060 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 3b8d74761b0b4fd0a22021d3870ec05c (table default.Table_0cc93ce7fcfb44089209422666978c62 [id=c0cd199f74494deda9d088345c4c90a3]) successfully deleted
I20260924 00:21:16.286484 11040 ts_tablet_manager.cc:1397] T 57ae3d10846a4aaa84eba81b13aafd44 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.040s user 0.008s sys 0.000s
I20260924 00:21:16.285540 11892 raft_consensus.cc:482] T 1d5a76bc420b41098d4ebd3dc59aa1fc P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:16.289287 11892 raft_consensus.cc:504] T 1d5a76bc420b41098d4ebd3dc59aa1fc P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.290323 11716 tablet_replica.cc:330] T e937cee22326474598cef5c1d09b59c2 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:16.291340 11892 leader_election.cc:288] T 1d5a76bc420b41098d4ebd3dc59aa1fc P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:16.290987 11040 raft_consensus.cc:348] T 57ae3d10846a4aaa84eba81b13aafd44 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.290954 11716 raft_consensus.cc:2229] T e937cee22326474598cef5c1d09b59c2 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:16.291702 11040 raft_consensus.cc:374] T 57ae3d10846a4aaa84eba81b13aafd44 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:16.291940 11040 raft_consensus.cc:729] T 57ae3d10846a4aaa84eba81b13aafd44 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:16.291994 11716 raft_consensus.cc:2258] T e937cee22326474598cef5c1d09b59c2 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:16.292459 11040 consensus_queue.cc:260] T 57ae3d10846a4aaa84eba81b13aafd44 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.293515 11716 ts_tablet_manager.cc:1905] T e937cee22326474598cef5c1d09b59c2 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:16.294502 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d5a76bc420b41098d4ebd3dc59aa1fc" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:16.300312 10592 leader_election.cc:341] T 1d5a76bc420b41098d4ebd3dc59aa1fc P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:16.295513 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d5a76bc420b41098d4ebd3dc59aa1fc" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:16.284333 11039 consensus_queue.cc:260] T 28be3abcaff64a3d97fad866472b41be P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.309437 10942 raft_consensus.cc:2454] T 1d5a76bc420b41098d4ebd3dc59aa1fc P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b10a9082a37b46f09d581944914c1670 in term 0.
I20260924 00:21:16.310770 11039 ts_tablet_manager.cc:1428] T 28be3abcaff64a3d97fad866472b41be P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.044s user 0.005s sys 0.000s
I20260924 00:21:16.310667 10589 leader_election.cc:302] T 1d5a76bc420b41098d4ebd3dc59aa1fc P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08
I20260924 00:21:16.311693 11892 raft_consensus.cc:2781] T 1d5a76bc420b41098d4ebd3dc59aa1fc P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:16.312045 11892 raft_consensus.cc:482] T 1d5a76bc420b41098d4ebd3dc59aa1fc P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:16.312398 11892 raft_consensus.cc:3037] T 1d5a76bc420b41098d4ebd3dc59aa1fc P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:16.288293 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_c879109e72984beb85e10878ed02c18f [id=0912091b30374df895fec45a7eafe4cc]
I20260924 00:21:16.322357 11719 ts_tablet_manager.cc:1918] T 5107b763acf04df88553ba0af2757570 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:16.322710 11719 log.cc:1196] T 5107b763acf04df88553ba0af2757570 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/5107b763acf04df88553ba0af2757570
I20260924 00:21:16.323257 11719 ts_tablet_manager.cc:1939] T 5107b763acf04df88553ba0af2757570 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:16.324496 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_b918cc5bd6a94a2a91b42dc02bb68be3 [id=09c0270c40904e3089ed7f4a0609bab6]
I20260924 00:21:16.325593 11799 raft_consensus.cc:482] T f85ac5b0fd2348d88e1076b781424ab3 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:16.326037 11799 raft_consensus.cc:504] T f85ac5b0fd2348d88e1076b781424ab3 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.327795 11799 leader_election.cc:288] T f85ac5b0fd2348d88e1076b781424ab3 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:16.328199 11889 raft_consensus.cc:482] T cf36d53828eb421d87b52b3e4cb1d748 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:16.328572 11889 raft_consensus.cc:504] T cf36d53828eb421d87b52b3e4cb1d748 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.329172 10653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f85ac5b0fd2348d88e1076b781424ab3" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:16.329701 10653 raft_consensus.cc:2454] T f85ac5b0fd2348d88e1076b781424ab3 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
I20260924 00:21:16.330384 11889 leader_election.cc:288] T cf36d53828eb421d87b52b3e4cb1d748 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:16.331632 10876 leader_election.cc:302] T f85ac5b0fd2348d88e1076b781424ab3 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:16.332304 11799 raft_consensus.cc:2781] T f85ac5b0fd2348d88e1076b781424ab3 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:16.332602 11799 raft_consensus.cc:482] T f85ac5b0fd2348d88e1076b781424ab3 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:16.332912 11799 raft_consensus.cc:3037] T f85ac5b0fd2348d88e1076b781424ab3 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:16.333267 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f85ac5b0fd2348d88e1076b781424ab3" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:16.335139 10878 leader_election.cc:341] T f85ac5b0fd2348d88e1076b781424ab3 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
W20260924 00:21:16.337723 10733 leader_election.cc:341] T cf36d53828eb421d87b52b3e4cb1d748 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: cf36d53828eb421d87b52b3e4cb1d748
I20260924 00:21:16.338385 11892 raft_consensus.cc:504] T 1d5a76bc420b41098d4ebd3dc59aa1fc P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.339736 11799 raft_consensus.cc:504] T f85ac5b0fd2348d88e1076b781424ab3 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.342460 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f85ac5b0fd2348d88e1076b781424ab3" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
W20260924 00:21:16.343686 10878 leader_election.cc:341] T f85ac5b0fd2348d88e1076b781424ab3 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:16.344120 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_9eeed352b33c49649d52ad128cb67203 [id=0b05d4353e924845834050bc136e812a]
I20260924 00:21:16.344728 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d5a76bc420b41098d4ebd3dc59aa1fc" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:21:16.345439 10384 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [term 3 FOLLOWER]: Refusing update from remote peer 1d68651913494ce8afee3c136b2da9cc: Log matching property violated. Preceding OpId in replica: term: 2 index: 453. Preceding OpId from leader: term: 3 index: 454. (index mismatch)
I20260924 00:21:16.345827 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_f8be74da4e7043749c326a27f9bc4bdf [id=0b35a077ccef43beba6f4e048b1ca46e]
W20260924 00:21:16.346593 10592 leader_election.cc:341] T 1d5a76bc420b41098d4ebd3dc59aa1fc P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:16.347465 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_e06918102eb24c628d528f20522dd2a4 [id=0b40ac1ec523496badcb50a185abeaf6]
I20260924 00:21:16.347604 11881 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 454, Last known committed idx: 453, Time since last communication: 0.000s
I20260924 00:21:16.353591 11892 leader_election.cc:288] T 1d5a76bc420b41098d4ebd3dc59aa1fc P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Requested vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:16.354525 10939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d5a76bc420b41098d4ebd3dc59aa1fc" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef"
I20260924 00:21:16.354866 11039 tablet_bootstrap.cc:492] T d4e120acff3e4c30aed813b0b24fbfee P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:16.355051 10939 raft_consensus.cc:3037] T 1d5a76bc420b41098d4ebd3dc59aa1fc P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:16.356855 11040 ts_tablet_manager.cc:1428] T 57ae3d10846a4aaa84eba81b13aafd44 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.068s user 0.005s sys 0.000s
W20260924 00:21:16.359670 10732 leader_election.cc:341] T cf36d53828eb421d87b52b3e4cb1d748 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Not found: Tablet not found: cf36d53828eb421d87b52b3e4cb1d748
I20260924 00:21:16.360061 10732 leader_election.cc:302] T cf36d53828eb421d87b52b3e4cb1d748 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08; no voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:16.364301 11040 tablet_bootstrap.cc:492] T 98926dd3850d40938b67779d5fd9a1db P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:16.366452 11885 raft_consensus.cc:2726] T cf36d53828eb421d87b52b3e4cb1d748 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:16.368013 11041 tablet_bootstrap.cc:492] T 28be3abcaff64a3d97fad866472b41be P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:16.368397 11041 ts_tablet_manager.cc:1397] T 28be3abcaff64a3d97fad866472b41be P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.097s user 0.006s sys 0.002s
I20260924 00:21:16.370433 11041 raft_consensus.cc:348] T 28be3abcaff64a3d97fad866472b41be P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.371043 11041 raft_consensus.cc:374] T 28be3abcaff64a3d97fad866472b41be P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:16.371280 11041 raft_consensus.cc:729] T 28be3abcaff64a3d97fad866472b41be P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:16.379441 11041 consensus_queue.cc:260] T 28be3abcaff64a3d97fad866472b41be P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.382210 11886 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d68651913494ce8afee3c136b2da9cc. Latest consensus state: current_term: 3 leader_uuid: "1d68651913494ce8afee3c136b2da9cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } } }
I20260924 00:21:16.382861 11041 ts_tablet_manager.cc:1428] T 28be3abcaff64a3d97fad866472b41be P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.014s user 0.001s sys 0.005s
I20260924 00:21:16.382943 11886 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:21:16.384196 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_6453acb2750442d0a172188dbc89d856 [id=0dd7e590773f4405bfb9c2a5060b3413]
I20260924 00:21:16.400862 11799 leader_election.cc:288] T f85ac5b0fd2348d88e1076b781424ab3 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:16.401460 10653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f85ac5b0fd2348d88e1076b781424ab3" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:21:16.402062 10653 raft_consensus.cc:3037] T f85ac5b0fd2348d88e1076b781424ab3 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:16.400449 10939 raft_consensus.cc:2454] T 1d5a76bc420b41098d4ebd3dc59aa1fc P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b10a9082a37b46f09d581944914c1670 in term 1.
I20260924 00:21:16.410239 10589 leader_election.cc:302] T 1d5a76bc420b41098d4ebd3dc59aa1fc P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08
I20260924 00:21:16.411242 11854 raft_consensus.cc:2781] T 1d5a76bc420b41098d4ebd3dc59aa1fc P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:16.415462 10653 raft_consensus.cc:2454] T f85ac5b0fd2348d88e1076b781424ab3 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 1.
I20260924 00:21:16.416538 10876 leader_election.cc:302] T f85ac5b0fd2348d88e1076b781424ab3 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08
I20260924 00:21:16.416916 11716 ts_tablet_manager.cc:1918] T e937cee22326474598cef5c1d09b59c2 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:16.417296 11716 log.cc:1196] T e937cee22326474598cef5c1d09b59c2 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/e937cee22326474598cef5c1d09b59c2
I20260924 00:21:16.417263 11808 raft_consensus.cc:2781] T f85ac5b0fd2348d88e1076b781424ab3 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:16.417692 11808 raft_consensus.cc:686] T f85ac5b0fd2348d88e1076b781424ab3 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Becoming Leader. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Running, Role: LEADER
I20260924 00:21:16.418418 11808 consensus_queue.cc:237] T f85ac5b0fd2348d88e1076b781424ab3 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.418946 11716 ts_tablet_manager.cc:1939] T e937cee22326474598cef5c1d09b59c2 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:16.425794 11854 raft_consensus.cc:686] T 1d5a76bc420b41098d4ebd3dc59aa1fc P b10a9082a37b46f09d581944914c1670 [term 1 LEADER]: Becoming Leader. State: Replica: b10a9082a37b46f09d581944914c1670, State: Running, Role: LEADER
I20260924 00:21:16.425895 11716 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35996 (request call id 334) took 2472 ms. Trace:
I20260924 00:21:16.426172 11716 rpcz_store.cc:276] 0924 00:21:13.953591 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:13.954591 (+ 1000us) service_pool.cc:227] Handling call
0924 00:21:16.290180 (+2335589us) ts_tablet_manager.cc:1145] Deleting tablet e937cee22326474598cef5c1d09b59c2
0924 00:21:16.290220 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:16.322730 (+ 32510us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:16.411986 (+ 89256us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:16.425837 (+ 13851us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2335063}
I20260924 00:21:16.427184 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet e937cee22326474598cef5c1d09b59c2 (table default.Table_d2a991e0c92e4c73b70bfbfb8bea5d58 [id=54d197b284b34dc0a5eea440b0b1e5de]) successfully deleted
I20260924 00:21:16.426652 11854 consensus_queue.cc:237] T 1d5a76bc420b41098d4ebd3dc59aa1fc P b10a9082a37b46f09d581944914c1670 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.428463 11716 tablet_replica.cc:330] T 675b9e33edcf4d4ea7fa428b476e3945 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:16.429188 11716 raft_consensus.cc:2229] T 675b9e33edcf4d4ea7fa428b476e3945 P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:21:16.429911 11716 pending_rounds.cc:62] T 675b9e33edcf4d4ea7fa428b476e3945 P 48fbd603abe146c585d15196e1b5be08: Trying to abort 1 pending ops.
I20260924 00:21:16.430148 11716 pending_rounds.cc:68] T 675b9e33edcf4d4ea7fa428b476e3945 P 48fbd603abe146c585d15196e1b5be08: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332697188320509952 op_type: NO_OP noop_request { }
I20260924 00:21:16.430498 11716 raft_consensus.cc:2866] T 675b9e33edcf4d4ea7fa428b476e3945 P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260924 00:21:16.430799 11716 raft_consensus.cc:2258] T 675b9e33edcf4d4ea7fa428b476e3945 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:16.432451 11719 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 325) took 2672 ms. Trace:
I20260924 00:21:16.432557 11716 ts_tablet_manager.cc:1905] T 675b9e33edcf4d4ea7fa428b476e3945 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:16.435631 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_aceb5020c4374a028638d9931d6665c9 [id=100796b0a2be4926b6fca423e2b164fa]
I20260924 00:21:16.440089 11040 tablet_bootstrap.cc:654] T 98926dd3850d40938b67779d5fd9a1db P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:16.438300 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_204d831d7f94465c8b58927b4988ef31 [id=1105b489035347609ca1fecc85d2fae3]
I20260924 00:21:16.450538 11719 rpcz_store.cc:276] 0924 00:21:13.759981 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:13.780073 (+ 20092us) service_pool.cc:227] Handling call
0924 00:21:16.160897 (+2380824us) ts_tablet_manager.cc:1145] Deleting tablet 5107b763acf04df88553ba0af2757570
0924 00:21:16.160958 (+ 61us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:16.272247 (+111289us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:16.322163 (+ 49916us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:16.432395 (+110232us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2287088}
I20260924 00:21:16.453544 11508 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 3 FOLLOWER]: Refusing update from remote peer 1d68651913494ce8afee3c136b2da9cc: Log matching property violated. Preceding OpId in replica: term: 2 index: 322. Preceding OpId from leader: term: 3 index: 454. (index mismatch)
I20260924 00:21:16.445631 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_5816c202c13e439cbd04eb58f456ebb5 [id=136a32ddaf2a4d2fbdddecf3ea95970c]
I20260924 00:21:16.455057 11881 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 454, Last known committed idx: 321, Time since last communication: 0.000s
I20260924 00:21:16.456105 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_dae0ffc8534047588f6f7cb13a610af5 [id=138be013fbcb4f01a6358d062200fe8f]
I20260924 00:21:16.464011 11886 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "1d68651913494ce8afee3c136b2da9cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } } }
I20260924 00:21:16.464704 11886 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5aa3debe1cd45e19a6bbd0070913ac3 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:21:16.466018 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_81b3a0208acd453c8a35957f47f25ff3 [id=14d056bc2e7849ebb6e6ca7966e11324]
I20260924 00:21:16.469154 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_045585fb6067443fbd323a3bbc688220 [id=1c3a915542a1470785614e97b5779149]
I20260924 00:21:16.434684 11041 tablet_bootstrap.cc:492] T d32a0c74b527451fb723e3851d5d055b P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:16.479527 11902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "1d68651913494ce8afee3c136b2da9cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } } }
I20260924 00:21:16.480381 11902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc [sys.catalog]: This master's current role is: LEADER
I20260924 00:21:16.481402 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_eb75a8c5ce6e48b2ba6c7b79990467ef [id=21a378fad88643b79609e92dc1b96bff]
I20260924 00:21:16.481822 11041 tablet_bootstrap.cc:654] T d32a0c74b527451fb723e3851d5d055b P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:16.483883 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_16ca7c156bce492ca79eec1b9881fff6 [id=222d2010881c4b5c9635aa3c28adc72b]
I20260924 00:21:16.499971 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_b65a4b1360e74260a30a4d0032e3c025 [id=244f1f1c84a14de9950e855b585177fe]
I20260924 00:21:16.501820 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_97629e88aeda47a880934c90cc33aaf6 [id=2613a8863ddb477498b493dfcc76114a]
I20260924 00:21:16.502343 11716 ts_tablet_manager.cc:1918] T 675b9e33edcf4d4ea7fa428b476e3945 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:16.502692 11716 log.cc:1196] T 675b9e33edcf4d4ea7fa428b476e3945 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/675b9e33edcf4d4ea7fa428b476e3945
I20260924 00:21:16.503240 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_99f14ba135cc459883dbb7c85dc21e30 [id=267c2c9545e547199df03ec06dda22be]
I20260924 00:21:16.503351 11716 ts_tablet_manager.cc:1939] T 675b9e33edcf4d4ea7fa428b476e3945 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:16.505097 11716 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35996 (request call id 335) took 2551 ms. Trace:
I20260924 00:21:16.505376 11716 rpcz_store.cc:276] 0924 00:21:13.953860 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:13.957048 (+ 3188us) service_pool.cc:227] Handling call
0924 00:21:16.428312 (+2471264us) ts_tablet_manager.cc:1145] Deleting tablet 675b9e33edcf4d4ea7fa428b476e3945
0924 00:21:16.428353 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:16.490907 (+ 62554us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:16.502131 (+ 11224us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:16.505047 (+ 2916us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2470785}
I20260924 00:21:16.506321 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 675b9e33edcf4d4ea7fa428b476e3945 (table default.Table_ee73693b7a6e46a3af53f7828da00f9e [id=9359b05533064c38b9bd589c0bb06471]) successfully deleted
I20260924 00:21:16.510147 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet 5107b763acf04df88553ba0af2757570 (table default.Table_e412cbac77574c5aafc587eaa698b633 [id=667f4349af3f4eb28461e33f65beeb1c]) successfully deleted
I20260924 00:21:16.511116 11719 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 331) took 2557 ms. Trace:
I20260924 00:21:16.513485 11719 rpcz_store.cc:276] 0924 00:21:13.953283 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:13.983620 (+ 30337us) service_pool.cc:227] Handling call
0924 00:21:16.510804 (+2527184us) ts_tablet_manager.cc:1145] Deleting tablet 1492ece2346d461ea57096a897daf83f
0924 00:21:16.510845 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:16.511062 (+ 217us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2526526}
I20260924 00:21:16.517338 11716 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35996 (request call id 336) took 2559 ms. Trace:
I20260924 00:21:16.517654 11716 rpcz_store.cc:276] 0924 00:21:13.957872 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:13.957959 (+ 87us) service_pool.cc:227] Handling call
0924 00:21:16.516996 (+2559037us) ts_tablet_manager.cc:1145] Deleting tablet 1492ece2346d461ea57096a897daf83f
0924 00:21:16.517041 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:16.517285 (+ 244us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2554177}
W20260924 00:21:16.518623 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 1492ece2346d461ea57096a897daf83f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1492ece2346d461ea57096a897daf83f already in progress: creating tablet
I20260924 00:21:16.518810 11716 tablet_replica.cc:330] T 85a0ad61b7cb4e11b30e84ddc4655b64 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:16.519445 11716 raft_consensus.cc:2229] T 85a0ad61b7cb4e11b30e84ddc4655b64 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:16.519886 11716 raft_consensus.cc:2258] T 85a0ad61b7cb4e11b30e84ddc4655b64 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:16.524519 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_a214c5e4b9584a1fba04c74e906fe4d3 [id=28864c6aa1f143bfb7b49d98bff16f8d]
I20260924 00:21:16.526346 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_b89f627e972b4c3e841bdb133667aa66 [id=2dd95b399ca84b569e46218f207f5950]
I20260924 00:21:16.530989 11039 tablet_bootstrap.cc:654] T d4e120acff3e4c30aed813b0b24fbfee P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:16.533494 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_d775e9e0f1614f9fbbae3dfd8554f4c2 [id=2e31f472d7504629b66c629cb0125d24]
I20260924 00:21:16.535228 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_a5d798f7819e403a96f27e98cb0af2bb [id=30a3f7c8fb9c46cf8b883eedae25d938]
W20260924 00:21:16.536263 10359 catalog_manager.cc:4821] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): delete failed for tablet 1492ece2346d461ea57096a897daf83f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1492ece2346d461ea57096a897daf83f already in progress: creating tablet
I20260924 00:21:16.537264 11719 tablet_replica.cc:330] T cd30f52583124543b5d616ef94b31831 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:16.538761 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_0a7f75ce29d74b7da36e39b072bffccc [id=332608cae68d4f51aff9ab21f3bb9e12]
I20260924 00:21:16.545526 11719 raft_consensus.cc:2229] T cd30f52583124543b5d616ef94b31831 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:16.546029 11719 raft_consensus.cc:2258] T cd30f52583124543b5d616ef94b31831 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:16.546190 11040 tablet_bootstrap.cc:492] T 98926dd3850d40938b67779d5fd9a1db P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:16.546568 11040 ts_tablet_manager.cc:1397] T 98926dd3850d40938b67779d5fd9a1db P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.182s user 0.008s sys 0.000s
I20260924 00:21:16.547706 11719 ts_tablet_manager.cc:1905] T cd30f52583124543b5d616ef94b31831 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:16.548629 11040 raft_consensus.cc:348] T 98926dd3850d40938b67779d5fd9a1db P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.549329 11040 raft_consensus.cc:374] T 98926dd3850d40938b67779d5fd9a1db P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:16.549557 11040 raft_consensus.cc:729] T 98926dd3850d40938b67779d5fd9a1db P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:16.541839 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_f1ae1e8ac652481680491d0da3355484 [id=36d61d1504cf495a9643bad809812d43]
I20260924 00:21:16.550151 11040 consensus_queue.cc:260] T 98926dd3850d40938b67779d5fd9a1db P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.551976 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_a85971df84464ddd9ca760e1f0e2ec3f [id=38ce54e1abbf493e85934dfb3232e881]
I20260924 00:21:16.553480 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_35151b4e43bd4e3bbf312c8f1c619e91 [id=3bdd316445814324a47269d8249a2089]
I20260924 00:21:16.555095 11041 tablet_bootstrap.cc:492] T d32a0c74b527451fb723e3851d5d055b P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:16.555060 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_cbd6d689c0a149d6a22cc163d3efb580 [id=3c20d5d6a1b3460882aa8f3f1bb71184]
I20260924 00:21:16.555642 11041 ts_tablet_manager.cc:1397] T d32a0c74b527451fb723e3851d5d055b P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.121s user 0.008s sys 0.002s
I20260924 00:21:16.556600 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_382099cf77a24bb8bd6fe084926431c9 [id=3db896d304514ead8dfe726c402a6fcd]
I20260924 00:21:16.557706 11716 ts_tablet_manager.cc:1905] T 85a0ad61b7cb4e11b30e84ddc4655b64 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:16.557816 11041 raft_consensus.cc:348] T d32a0c74b527451fb723e3851d5d055b P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.558122 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_d853fb40e0554242976678966283153e [id=3e6243f4f29c41db9fc70e43473ca92a]
I20260924 00:21:16.558498 11041 raft_consensus.cc:374] T d32a0c74b527451fb723e3851d5d055b P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:16.558805 11041 raft_consensus.cc:729] T d32a0c74b527451fb723e3851d5d055b P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:16.559455 11041 consensus_queue.cc:260] T d32a0c74b527451fb723e3851d5d055b P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.569058 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_36c8c183203d418c9a15405725842b64 [id=3f954b8033264f41b0605ee408d21ed0]
I20260924 00:21:16.581754 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_9e4907fea442455b9ac9c793dfabf496 [id=41a14bd521754a1e8fdd166c7a83bd62]
I20260924 00:21:16.583436 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_79486769678c44fe98623f5aa06975bb [id=423a3ef82b0f488fa8fb082d4e6f63a3]
I20260924 00:21:16.584833 11040 ts_tablet_manager.cc:1428] T 98926dd3850d40938b67779d5fd9a1db P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.038s user 0.006s sys 0.000s
I20260924 00:21:16.585521 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_63927aa1872e41da917d6b92c8acb310 [id=485c42c942b74abbb32bea36d25bc184]
I20260924 00:21:16.587065 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_396157d80ec44f329a275ccd3167bcb7 [id=4a372ce51c6b4915946217dec64719fa]
I20260924 00:21:16.587284 11799 raft_consensus.cc:482] T 2e839fd66fed4472a0892baa9aa85c54 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:16.587736 11799 raft_consensus.cc:504] T 2e839fd66fed4472a0892baa9aa85c54 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.589524 11799 leader_election.cc:288] T 2e839fd66fed4472a0892baa9aa85c54 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:16.590408 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e839fd66fed4472a0892baa9aa85c54" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:16.591670 10878 leader_election.cc:341] T 2e839fd66fed4472a0892baa9aa85c54 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:16.593276 10653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e839fd66fed4472a0892baa9aa85c54" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:16.593827 10653 raft_consensus.cc:2454] T 2e839fd66fed4472a0892baa9aa85c54 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
I20260924 00:21:16.594914 10876 leader_election.cc:302] T 2e839fd66fed4472a0892baa9aa85c54 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08
I20260924 00:21:16.595867 11799 raft_consensus.cc:2781] T 2e839fd66fed4472a0892baa9aa85c54 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:16.596180 11799 raft_consensus.cc:482] T 2e839fd66fed4472a0892baa9aa85c54 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:16.596459 11799 raft_consensus.cc:3037] T 2e839fd66fed4472a0892baa9aa85c54 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:16.597461 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_e724566b223445b7b3988eeb3e811bcd [id=4b08b25314a64fdd8313975f514fb10a]
I20260924 00:21:16.601181 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_66a5eb5613d743c1b96af8b5a252fd85 [id=4dbb68e0a32349ad8da2945db650584d]
I20260924 00:21:16.605598 11799 raft_consensus.cc:504] T 2e839fd66fed4472a0892baa9aa85c54 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.608107 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e839fd66fed4472a0892baa9aa85c54" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
W20260924 00:21:16.609299 10878 leader_election.cc:341] T 2e839fd66fed4472a0892baa9aa85c54 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:16.611057 10653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e839fd66fed4472a0892baa9aa85c54" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:21:16.611676 10653 raft_consensus.cc:3037] T 2e839fd66fed4472a0892baa9aa85c54 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:16.602545 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_bf61c7623556425688b99c1c373e4879 [id=4f70e833d7f74d91af0faa01ed52f78b]
I20260924 00:21:16.618194 10653 raft_consensus.cc:2454] T 2e839fd66fed4472a0892baa9aa85c54 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 1.
I20260924 00:21:16.618466 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_d90e8159c7b34d59bcde6044c2e70bb8 [id=526561ff42b043169b418f9996d8006b]
I20260924 00:21:16.619274 10876 leader_election.cc:302] T 2e839fd66fed4472a0892baa9aa85c54 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08
I20260924 00:21:16.619977 11808 raft_consensus.cc:2781] T 2e839fd66fed4472a0892baa9aa85c54 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:16.624806 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_8823ba2975054860816e34a4fa104be8 [id=556a0c70dd484c5aaa120eb5f0dfe8e1]
I20260924 00:21:16.626348 11041 ts_tablet_manager.cc:1428] T d32a0c74b527451fb723e3851d5d055b P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.070s user 0.005s sys 0.000s
I20260924 00:21:16.627228 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_ace1c8c32aaa43d6a39b2e3caba246be [id=5735e1d6c05e474e95239f5dc4b44a02]
I20260924 00:21:16.628609 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_aa256627fd99496e988fab956cde2057 [id=579f89b1e70f4dc2b2080868adfeea76]
I20260924 00:21:16.630061 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_d1104bf9437443d48b9e769fcf06d5ec [id=5816a524b40d48629911e69babd7a80a]
I20260924 00:21:16.630987 11808 raft_consensus.cc:686] T 2e839fd66fed4472a0892baa9aa85c54 P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Becoming Leader. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Running, Role: LEADER
I20260924 00:21:16.631390 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_9f2df24361da4f9a9340f6f8673043ea [id=59ec3e94e2f44150994a5ffc1d9fa8f4]
I20260924 00:21:16.632763 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_6cebadb2663346f7b016f42a3ed0d1a5 [id=5b5b6b3fe58b433baf867711db148618]
I20260924 00:21:16.634030 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_e15931b8ec344adebc1ba60fa4be6f9b [id=5d3eb8f4da1b4b69ac59d69af11e5e46]
I20260924 00:21:16.635185 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_76ffb857cf034b6f80598edd90dad953 [id=5fb5e1e90c9440f385e7b04520bcba9c]
I20260924 00:21:16.636173 11808 consensus_queue.cc:237] T 2e839fd66fed4472a0892baa9aa85c54 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.640651 11041 tablet_bootstrap.cc:492] T 8e628abffd9545b391c25f83c9903732 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:16.638324 11039 tablet_bootstrap.cc:492] T d4e120acff3e4c30aed813b0b24fbfee P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:16.642396 11039 ts_tablet_manager.cc:1397] T d4e120acff3e4c30aed813b0b24fbfee P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.288s user 0.008s sys 0.003s
I20260924 00:21:16.645481 11799 leader_election.cc:288] T 2e839fd66fed4472a0892baa9aa85c54 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:16.647210 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_477c0f1cce9a4c089551e4e767ba7ae0 [id=6251651eb5944ee086dd564d86204148]
I20260924 00:21:16.647426 11040 tablet_bootstrap.cc:492] T 9319f9caadb04067a2e9700b5019ed37 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:16.648773 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_24047fc04a434959928e6148f187a672 [id=64a8cee150dd4ad4bba25adf7dbe36e5]
I20260924 00:21:16.660471 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_e412cbac77574c5aafc587eaa698b633 [id=667f4349af3f4eb28461e33f65beeb1c]
I20260924 00:21:16.662480 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_f24d66ccb67d4a5fb0c34a09c63e089c [id=6a53cdb2501c4276938cdeae3919624a]
I20260924 00:21:16.663738 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_6ba8cc42d97e4576a8c2021b39b78dbb [id=6ac68d6b800747d59768d4988137dae7]
I20260924 00:21:16.665091 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_b7b77b7460bf4db393567227b3536333 [id=6ca9fc0b6417483bacc49673da19e9c4]
I20260924 00:21:16.666887 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_2ff16b0fac094dc1865b40d6c91bbfdc [id=6fca68e9ecc44d71bae53204e1894542]
I20260924 00:21:16.668804 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_9c2aa717e62340c785be379c0dd5837c [id=737634d9f0f4413c8b6dcaa26de87e2d]
I20260924 00:21:16.669982 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_22740cf31fbe40a58c31252f63417ad3 [id=74b4a09bdb464f27ae5d19a6a53ece5f]
I20260924 00:21:16.671178 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_7a350a3ca82c427298dd0bedb9db3d60 [id=775fb1f73d864bac8c0265786727798c]
I20260924 00:21:16.672309 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_b07b221a38e14a93aa27e7c2dfe55549 [id=79232ff751084c1292ba92127d124c43]
I20260924 00:21:16.644562 11039 raft_consensus.cc:348] T d4e120acff3e4c30aed813b0b24fbfee P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:16.678846 11041 tablet_bootstrap.cc:654] T 8e628abffd9545b391c25f83c9903732 P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:16.678835 11039 raft_consensus.cc:374] T d4e120acff3e4c30aed813b0b24fbfee P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:16.679205 11039 raft_consensus.cc:729] T d4e120acff3e4c30aed813b0b24fbfee P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:16.679922 11039 consensus_queue.cc:260] T d4e120acff3e4c30aed813b0b24fbfee P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:16.683103 11039 ts_tablet_manager.cc:1428] T d4e120acff3e4c30aed813b0b24fbfee P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.040s user 0.005s sys 0.000s
I20260924 00:21:16.693291 11041 tablet_bootstrap.cc:492] T 8e628abffd9545b391c25f83c9903732 P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:16.693674 11041 ts_tablet_manager.cc:1397] T 8e628abffd9545b391c25f83c9903732 P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.053s user 0.009s sys 0.000s
I20260924 00:21:16.694027 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_f9897a81fc084a25ae405522cf5c7ef9 [id=7a6f8b0af8484d8b80c02e80027db9d4]
I20260924 00:21:16.696084 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_82b79ff8ffca4a7880702b55cd10e393 [id=7e160144b1704e27a1d7f729c7a04049]
I20260924 00:21:16.701212 10367 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 6bcfd729e9bd4040b3384ab24164e80d (table default.Table_9670783a92074b639ecb0d6c374811e5 [id=df2a5a74db65431c9ecb750184f64709]). Expected version 1 got 0
I20260924 00:21:16.702059 10367 catalog_manager.cc:5587] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301) does not have the latest schema for tablet 4c2a729a802e4abe9ab4f989a40198be (table default.Table_5816c202c13e439cbd04eb58f456ebb5 [id=136a32ddaf2a4d2fbdddecf3ea95970c]). Expected version 2 got 0
W20260924 00:21:16.708009 10367 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.4.67:49569 (request call id 74) took 1088 ms (client timeout 1000 ms). Trace:
I20260924 00:21:16.708601 10908 tablet_service.cc:1514] Processing DeleteTablet for tablet 57ae3d10846a4aaa84eba81b13aafd44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:21:09 UTC) from {username='slave'} at 127.0.0.1:35358
W20260924 00:21:16.708590 10367 rpcz_store.cc:269] 0924 00:21:15.619906 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:15.620024 (+ 118us) service_pool.cc:227] Handling call
0924 00:21:16.707944 (+1087920us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":9}
I20260924 00:21:16.652135 11040 tablet_bootstrap.cc:654] T 9319f9caadb04067a2e9700b5019ed37 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:16.717939 11039 tablet_bootstrap.cc:492] T a53f06276dbb4d9f97a154a8ceb0278d P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:16.726374 11918 tablet_replica.cc:330] T 57ae3d10846a4aaa84eba81b13aafd44 P c8626e300d0644b0908797ad61bd7cef: stopping tablet replica
I20260924 00:21:16.727367 11918 raft_consensus.cc:2229] T 57ae3d10846a4aaa84eba81b13aafd44 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:16.727907 11918 raft_consensus.cc:2258] T 57ae3d10846a4aaa84eba81b13aafd44 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:16.729686 11918 ts_tablet_manager.cc:1905] T 57ae3d10846a4aaa84eba81b13aafd44 P c8626e300d0644b0908797ad61bd7cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:16.731248 11040 tablet_bootstrap.cc:492] T 9319f9caadb04067a2e9700b5019ed37 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:16.731702 11040 ts_tablet_manager.cc:1397] T 9319f9caadb04067a2e9700b5019ed37 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.084s user 0.009s sys 0.000s
I20260924 00:21:16.733600 11041 raft_consensus.cc:348] T 8e628abffd9545b391c25f83c9903732 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.734241 11041 raft_consensus.cc:374] T 8e628abffd9545b391c25f83c9903732 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:16.734468 11041 raft_consensus.cc:729] T 8e628abffd9545b391c25f83c9903732 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:16.735080 11041 consensus_queue.cc:260] T 8e628abffd9545b391c25f83c9903732 P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.741873 11854 raft_consensus.cc:482] T 28be3abcaff64a3d97fad866472b41be P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:16.742267 11854 raft_consensus.cc:504] T 28be3abcaff64a3d97fad866472b41be P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.744058 11854 leader_election.cc:288] T 28be3abcaff64a3d97fad866472b41be P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:16.734227 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_e735211dc38c444da1b3af42f354c740 [id=8031d25aceec4ca48e8b29e7d75e5320]
I20260924 00:21:16.746863 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28be3abcaff64a3d97fad866472b41be" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:16.748144 10592 leader_election.cc:341] T 28be3abcaff64a3d97fad866472b41be P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:16.748633 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_e207dff5fb014842a20e58edf524849e [id=806be42cdd1b43f788520b28ecaf41a7]
I20260924 00:21:16.754781 10795 raft_consensus.cc:3037] T 4c2a729a802e4abe9ab4f989a40198be P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:16.759166 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_a9562c9df88a4874901902e8673278b5 [id=818963f97e3d4ae8a8e1d87e8511725d]
I20260924 00:21:16.745510 10939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28be3abcaff64a3d97fad866472b41be" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:16.758046 11039 tablet_bootstrap.cc:654] T a53f06276dbb4d9f97a154a8ceb0278d P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:16.761245 10939 raft_consensus.cc:2454] T 28be3abcaff64a3d97fad866472b41be P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b10a9082a37b46f09d581944914c1670 in term 0.
I20260924 00:21:16.762450 10589 leader_election.cc:302] T 28be3abcaff64a3d97fad866472b41be P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08
I20260924 00:21:16.763512 11854 raft_consensus.cc:2781] T 28be3abcaff64a3d97fad866472b41be P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:16.763450 11808 consensus_queue.cc:1035] T 4c2a729a802e4abe9ab4f989a40198be P c8626e300d0644b0908797ad61bd7cef [LEADER]: Connected to new peer: Peer: permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:16.763831 11854 raft_consensus.cc:482] T 28be3abcaff64a3d97fad866472b41be P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:16.763995 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_69005ec88288442691befac5eff2adc7 [id=8308fedce3544a18912f343a3f4b519c]
I20260924 00:21:16.764109 11854 raft_consensus.cc:3037] T 28be3abcaff64a3d97fad866472b41be P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:16.770002 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_d1738e7eeecb4601a9402b1745b33bd3 [id=8875bd95730045a4b0bfc9176f0c023e]
I20260924 00:21:16.771240 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_4541883eff524699a7226b51ad3b9727 [id=896bcf73282349db8fab4222fc76ca71]
I20260924 00:21:16.771668 11854 raft_consensus.cc:504] T 28be3abcaff64a3d97fad866472b41be P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.774276 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28be3abcaff64a3d97fad866472b41be" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:21:16.774979 10939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28be3abcaff64a3d97fad866472b41be" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef"
E20260924 00:21:16.775210 10555 catalog_manager.cc:816] Error processing pending assignments: Illegal state: error persisting updated tablet metadata: replica b5aa3debe1cd45e19a6bbd0070913ac3 is not leader of this config: current role FOLLOWER
I20260924 00:21:16.775520 10939 raft_consensus.cc:3037] T 28be3abcaff64a3d97fad866472b41be P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
W20260924 00:21:16.777136 10592 leader_election.cc:341] T 28be3abcaff64a3d97fad866472b41be P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:16.780138 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_d02a9cefdc804218a76ef33174772e09 [id=9037f3cc30d440c39da4db959a76b52f]
I20260924 00:21:16.782024 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_ee73693b7a6e46a3af53f7828da00f9e [id=9359b05533064c38b9bd589c0bb06471]
I20260924 00:21:16.782258 10939 raft_consensus.cc:2454] T 28be3abcaff64a3d97fad866472b41be P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b10a9082a37b46f09d581944914c1670 in term 1.
I20260924 00:21:16.783234 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_b36ac15778564d608790f586457c6373 [id=93c1734de58245f8b1d5ab88ec9f0c8e]
I20260924 00:21:16.783511 10589 leader_election.cc:302] T 28be3abcaff64a3d97fad866472b41be P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08
I20260924 00:21:16.784188 11892 raft_consensus.cc:2781] T 28be3abcaff64a3d97fad866472b41be P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:16.784425 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_8ff1c7565eff465da9ae77704e862894 [id=94267d66c5a74dd38e9f005f77b1ee18]
I20260924 00:21:16.784613 11892 raft_consensus.cc:686] T 28be3abcaff64a3d97fad866472b41be P b10a9082a37b46f09d581944914c1670 [term 1 LEADER]: Becoming Leader. State: Replica: b10a9082a37b46f09d581944914c1670, State: Running, Role: LEADER
I20260924 00:21:16.785755 11719 ts_tablet_manager.cc:1918] T cd30f52583124543b5d616ef94b31831 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:16.785329 11892 consensus_queue.cc:237] T 28be3abcaff64a3d97fad866472b41be P b10a9082a37b46f09d581944914c1670 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.786158 11039 tablet_bootstrap.cc:492] T a53f06276dbb4d9f97a154a8ceb0278d P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:16.786520 11039 ts_tablet_manager.cc:1397] T a53f06276dbb4d9f97a154a8ceb0278d P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.069s user 0.002s sys 0.009s
I20260924 00:21:16.788653 11039 raft_consensus.cc:348] T a53f06276dbb4d9f97a154a8ceb0278d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:16.789598 11039 raft_consensus.cc:374] T a53f06276dbb4d9f97a154a8ceb0278d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:16.789841 11040 raft_consensus.cc:348] T 9319f9caadb04067a2e9700b5019ed37 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.790660 11040 raft_consensus.cc:374] T 9319f9caadb04067a2e9700b5019ed37 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:16.790932 11040 raft_consensus.cc:729] T 9319f9caadb04067a2e9700b5019ed37 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:16.791787 11040 consensus_queue.cc:260] T 9319f9caadb04067a2e9700b5019ed37 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.793927 11040 ts_tablet_manager.cc:1428] T 9319f9caadb04067a2e9700b5019ed37 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.062s user 0.005s sys 0.000s
I20260924 00:21:16.789839 11039 raft_consensus.cc:729] T a53f06276dbb4d9f97a154a8ceb0278d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:16.797576 11039 consensus_queue.cc:260] T a53f06276dbb4d9f97a154a8ceb0278d P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:16.809634 11039 ts_tablet_manager.cc:1428] T a53f06276dbb4d9f97a154a8ceb0278d P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.023s user 0.005s sys 0.002s
I20260924 00:21:16.817044 11854 leader_election.cc:288] T 28be3abcaff64a3d97fad866472b41be P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Requested vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:16.801230 11716 ts_tablet_manager.cc:1918] T 85a0ad61b7cb4e11b30e84ddc4655b64 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260924 00:21:16.821419 11041 ts_tablet_manager.cc:1428] T 8e628abffd9545b391c25f83c9903732 P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.127s user 0.006s sys 0.000s
I20260924 00:21:16.821792 11716 log.cc:1196] T 85a0ad61b7cb4e11b30e84ddc4655b64 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/85a0ad61b7cb4e11b30e84ddc4655b64
I20260924 00:21:16.822575 11716 ts_tablet_manager.cc:1939] T 85a0ad61b7cb4e11b30e84ddc4655b64 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:16.810510 10795 raft_consensus.cc:3037] T 98926dd3850d40938b67779d5fd9a1db P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:16.817090 11719 log.cc:1196] T cd30f52583124543b5d616ef94b31831 P b10a9082a37b46f09d581944914c1670: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/cd30f52583124543b5d616ef94b31831
I20260924 00:21:16.830061 11716 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35996 (request call id 339) took 2524 ms. Trace:
I20260924 00:21:16.830446 11716 rpcz_store.cc:276] 0924 00:21:14.306092 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:14.306173 (+ 81us) service_pool.cc:227] Handling call
0924 00:21:16.518679 (+2212506us) ts_tablet_manager.cc:1145] Deleting tablet 85a0ad61b7cb4e11b30e84ddc4655b64
0924 00:21:16.518716 (+ 37us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:16.701777 (+183061us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:16.744829 (+ 43052us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:16.830000 (+ 85171us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2143820}
I20260924 00:21:16.831043 11039 tablet_bootstrap.cc:492] T d32a0c74b527451fb723e3851d5d055b P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:16.831761 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 85a0ad61b7cb4e11b30e84ddc4655b64 (table default.Table_07390b277c404c7c9daecd17fe3dccca [id=12285914811f4220822b47a2a62b01ab]) successfully deleted
I20260924 00:21:16.832082 11716 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35996 (request call id 340) took 2525 ms. Trace:
I20260924 00:21:16.832342 11716 rpcz_store.cc:276] 0924 00:21:14.306793 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:14.307980 (+ 1187us) service_pool.cc:227] Handling call
0924 00:21:16.831767 (+2523787us) ts_tablet_manager.cc:1145] Deleting tablet 3dae8b8d7743416983e9e903ea57140d
0924 00:21:16.831804 (+ 37us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:16.832038 (+ 234us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2445562}
W20260924 00:21:16.833513 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 3dae8b8d7743416983e9e903ea57140d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3dae8b8d7743416983e9e903ea57140d already in progress: creating tablet
I20260924 00:21:16.833606 11716 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35996 (request call id 341) took 2512 ms. Trace:
I20260924 00:21:16.833844 11716 rpcz_store.cc:276] 0924 00:21:14.321305 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:14.321412 (+ 107us) service_pool.cc:227] Handling call
0924 00:21:16.833322 (+2511910us) ts_tablet_manager.cc:1145] Deleting tablet 52c46283934a4c2180f15c7fc38b91a9
0924 00:21:16.833346 (+ 24us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:16.833562 (+ 216us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2442350}
I20260924 00:21:16.834878 11040 tablet_bootstrap.cc:492] T 5107b763acf04df88553ba0af2757570 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
W20260924 00:21:16.835031 10361 catalog_manager.cc:4821] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): delete failed for tablet 52c46283934a4c2180f15c7fc38b91a9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 52c46283934a4c2180f15c7fc38b91a9 already in progress: creating tablet
I20260924 00:21:16.835996 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_aa8bce70b4eb4f9a957369558580a1a2 [id=95fb80b2ff8044809780c83dcdd07cf0]
I20260924 00:21:16.838615 11719 ts_tablet_manager.cc:1939] T cd30f52583124543b5d616ef94b31831 P b10a9082a37b46f09d581944914c1670: Deleting consensus metadata
I20260924 00:21:16.840431 11719 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46896 (request call id 335) took 2678 ms. Trace:
I20260924 00:21:16.841562 11719 rpcz_store.cc:276] 0924 00:21:14.162447 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:14.162697 (+ 250us) service_pool.cc:227] Handling call
0924 00:21:16.537118 (+2374421us) ts_tablet_manager.cc:1145] Deleting tablet cd30f52583124543b5d616ef94b31831
0924 00:21:16.537161 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:16.675737 (+138576us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:16.785543 (+109806us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:16.840378 (+ 54835us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2373910}
I20260924 00:21:16.842551 11719 tablet_replica.cc:330] T 85a0ad61b7cb4e11b30e84ddc4655b64 P b10a9082a37b46f09d581944914c1670: stopping tablet replica
I20260924 00:21:16.843109 10359 catalog_manager.cc:4841] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): tablet cd30f52583124543b5d616ef94b31831 (table default.Table_3078c59b2af247ab91ea35e4f8bf9b9c [id=89a81661022b40eda6eecd81008a3107]) successfully deleted
I20260924 00:21:16.843359 11039 tablet_bootstrap.cc:654] T d32a0c74b527451fb723e3851d5d055b P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:16.844592 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_cae9fb8a84b248ac96ee0c3ee9874324 [id=972f6bb2a2094f5a9a49ac19ff3e0f71]
I20260924 00:21:16.863266 11040 tablet_bootstrap.cc:654] T 5107b763acf04df88553ba0af2757570 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:16.863435 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_29d22b606c3a4d4ba2fce79bd410c4f4 [id=9839446621034266a1e4815b8f4863c2]
I20260924 00:21:16.843197 11719 raft_consensus.cc:2229] T 85a0ad61b7cb4e11b30e84ddc4655b64 P b10a9082a37b46f09d581944914c1670 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:21:16.865866 11719 raft_consensus.cc:2258] T 85a0ad61b7cb4e11b30e84ddc4655b64 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:16.866238 11854 raft_consensus.cc:482] T c7cb7c6892404c7fa593cdaaac67f800 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:16.866693 11854 raft_consensus.cc:504] T c7cb7c6892404c7fa593cdaaac67f800 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.867818 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_5cf852dcccf3498f96d0e5e8364d1a84 [id=a15434697c7b4707a0effb20cbb36609]
I20260924 00:21:16.869571 10794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7cb7c6892404c7fa593cdaaac67f800" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:16.870739 10592 leader_election.cc:341] T c7cb7c6892404c7fa593cdaaac67f800 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:16.871716 11854 leader_election.cc:288] T c7cb7c6892404c7fa593cdaaac67f800 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:16.872085 10939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7cb7c6892404c7fa593cdaaac67f800" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:16.872674 10939 raft_consensus.cc:2454] T c7cb7c6892404c7fa593cdaaac67f800 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b10a9082a37b46f09d581944914c1670 in term 0.
I20260924 00:21:16.873806 10589 leader_election.cc:302] T c7cb7c6892404c7fa593cdaaac67f800 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08
I20260924 00:21:16.874559 11039 tablet_bootstrap.cc:492] T d32a0c74b527451fb723e3851d5d055b P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:16.874742 11854 raft_consensus.cc:2781] T c7cb7c6892404c7fa593cdaaac67f800 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:16.874945 11039 ts_tablet_manager.cc:1397] T d32a0c74b527451fb723e3851d5d055b P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.044s user 0.005s sys 0.004s
I20260924 00:21:16.875200 11854 raft_consensus.cc:482] T c7cb7c6892404c7fa593cdaaac67f800 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:16.875555 11854 raft_consensus.cc:3037] T c7cb7c6892404c7fa593cdaaac67f800 P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:16.877053 11039 raft_consensus.cc:348] T d32a0c74b527451fb723e3851d5d055b P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.877673 11039 raft_consensus.cc:374] T d32a0c74b527451fb723e3851d5d055b P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:16.877903 11039 raft_consensus.cc:729] T d32a0c74b527451fb723e3851d5d055b P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:16.880136 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_3245e93d2b494073b531fd16a48f1244 [id=a33bb157c42440158194f77c2f9c07a7]
I20260924 00:21:16.881187 11039 consensus_queue.cc:260] T d32a0c74b527451fb723e3851d5d055b P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.883018 11039 ts_tablet_manager.cc:1428] T d32a0c74b527451fb723e3851d5d055b P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.008s user 0.004s sys 0.000s
I20260924 00:21:16.884039 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_a5a83435a7fb47ce8083022c0686b74a [id=a587a50b0c204843a41acacc45c2b150]
I20260924 00:21:16.885353 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_4c16876881e6429fbe20b9eb73c68d3f [id=a672b0959c50464eb890db93fd96c64c]
I20260924 00:21:16.887120 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_14437b13dbe34d5591c2a180c12c23b1 [id=aa1298d4fa2d40ac93e7c74077f294ce]
I20260924 00:21:16.889624 11039 tablet_bootstrap.cc:492] T d574642ebc6340fcb5e83c36804ee9f8 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:16.890386 11799 raft_consensus.cc:482] T d32a0c74b527451fb723e3851d5d055b P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:16.890817 11799 raft_consensus.cc:504] T d32a0c74b527451fb723e3851d5d055b P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.892486 11799 leader_election.cc:288] T d32a0c74b527451fb723e3851d5d055b P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:16.893956 10653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d32a0c74b527451fb723e3851d5d055b" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:16.894534 10653 raft_consensus.cc:2454] T d32a0c74b527451fb723e3851d5d055b P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
I20260924 00:21:16.895869 10876 leader_election.cc:302] T d32a0c74b527451fb723e3851d5d055b P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:16.897579 11799 raft_consensus.cc:2781] T d32a0c74b527451fb723e3851d5d055b P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:16.897883 11799 raft_consensus.cc:482] T d32a0c74b527451fb723e3851d5d055b P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:16.897948 11039 tablet_bootstrap.cc:654] T d574642ebc6340fcb5e83c36804ee9f8 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:16.898198 11799 raft_consensus.cc:3037] T d32a0c74b527451fb723e3851d5d055b P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:16.899540 10794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d32a0c74b527451fb723e3851d5d055b" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:16.902287 11716 tablet_replica.cc:330] T 717759eb61674c89b05b8d2b0dc34979 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
W20260924 00:21:16.902231 10878 leader_election.cc:341] T d32a0c74b527451fb723e3851d5d055b P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:16.906423 11041 tablet_bootstrap.cc:492] T 0de205104d7c4caf8f8f228e6a0d8c3a P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:16.910379 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_9608bad42525495eaa4df537b37969e7 [id=aa74d52589984aec8e6d296e5dffd623]
I20260924 00:21:16.911553 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_cd15e5e8e07947459840c3ed5dca1f1d [id=ad71d06959604cdf89d7dbe5716d2d08]
I20260924 00:21:16.913192 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_8f274639bca846bf89f8770c7de95055 [id=adababd4f08043c89167f5c28e00e6f0]
I20260924 00:21:16.916756 11039 tablet_bootstrap.cc:492] T d574642ebc6340fcb5e83c36804ee9f8 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:16.917146 11039 ts_tablet_manager.cc:1397] T d574642ebc6340fcb5e83c36804ee9f8 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.028s user 0.007s sys 0.002s
I20260924 00:21:16.918496 11799 raft_consensus.cc:504] T d32a0c74b527451fb723e3851d5d055b P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.902992 11716 raft_consensus.cc:2229] T 717759eb61674c89b05b8d2b0dc34979 P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:21:16.915416 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_5bdf50c2b69b4f3cadbb52a11cf10035 [id=b07edcb7ee5044c6a486984cc3efe917]
I20260924 00:21:16.920907 11719 ts_tablet_manager.cc:1905] T 85a0ad61b7cb4e11b30e84ddc4655b64 P b10a9082a37b46f09d581944914c1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:16.922508 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_e33891e1d1b745819f1114e54bded8bb [id=b382a8277aa3452990df0abe27a80cb2]
I20260924 00:21:16.923759 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_ceb867d167eb4df8bb2653ede8e37644 [id=b8ac1418c7d644b7b635d6cb28c100b6]
I20260924 00:21:16.924976 11716 pending_rounds.cc:62] T 717759eb61674c89b05b8d2b0dc34979 P 48fbd603abe146c585d15196e1b5be08: Trying to abort 1 pending ops.
I20260924 00:21:16.925186 11716 pending_rounds.cc:68] T 717759eb61674c89b05b8d2b0dc34979 P 48fbd603abe146c585d15196e1b5be08: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332697194606006272 op_type: NO_OP noop_request { }
I20260924 00:21:16.925478 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_a51a7905e6f848edb48e25b18d0ae196 [id=ba9dbd5eba46433d8166e91188c49599]
I20260924 00:21:16.925503 11716 raft_consensus.cc:2866] T 717759eb61674c89b05b8d2b0dc34979 P 48fbd603abe146c585d15196e1b5be08 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260924 00:21:16.931931 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_83435d6ddd64425390ab1b05337d2081 [id=bf5691bc3ec148eda2cb116bb7842cfc]
I20260924 00:21:16.933266 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_16819990f8b0403a8947a30c882feb5a [id=bf8333796bff4b25908865364e81bfec]
I20260924 00:21:16.919102 11039 raft_consensus.cc:348] T d574642ebc6340fcb5e83c36804ee9f8 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:16.933869 11039 raft_consensus.cc:374] T d574642ebc6340fcb5e83c36804ee9f8 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:16.934229 11039 raft_consensus.cc:729] T d574642ebc6340fcb5e83c36804ee9f8 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:16.934811 11039 consensus_queue.cc:260] T d574642ebc6340fcb5e83c36804ee9f8 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:16.936650 11039 ts_tablet_manager.cc:1428] T d574642ebc6340fcb5e83c36804ee9f8 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.019s user 0.004s sys 0.000s
I20260924 00:21:16.937877 11039 tablet_bootstrap.cc:492] T 0de205104d7c4caf8f8f228e6a0d8c3a P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:16.938443 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d32a0c74b527451fb723e3851d5d055b" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:21:16.943042 11039 tablet_bootstrap.cc:654] T 0de205104d7c4caf8f8f228e6a0d8c3a P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:16.948284 11928 raft_consensus.cc:482] T 57ae3d10846a4aaa84eba81b13aafd44 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:16.948467 11913 consensus_queue.cc:1035] T 98926dd3850d40938b67779d5fd9a1db P c8626e300d0644b0908797ad61bd7cef [LEADER]: Connected to new peer: Peer: permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:16.948922 11928 raft_consensus.cc:504] T 57ae3d10846a4aaa84eba81b13aafd44 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.951550 10939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57ae3d10846a4aaa84eba81b13aafd44" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
W20260924 00:21:16.953437 10732 leader_election.cc:341] T 57ae3d10846a4aaa84eba81b13aafd44 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Illegal state: Tablet not RUNNING: STOPPED
W20260924 00:21:16.954546 10733 leader_election.cc:341] T 57ae3d10846a4aaa84eba81b13aafd44 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Not found: Tablet not found: 57ae3d10846a4aaa84eba81b13aafd44
I20260924 00:21:16.954887 10733 leader_election.cc:302] T 57ae3d10846a4aaa84eba81b13aafd44 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08; no voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef
I20260924 00:21:16.955983 11928 leader_election.cc:288] T 57ae3d10846a4aaa84eba81b13aafd44 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
W20260924 00:21:16.956005 10878 leader_election.cc:341] T d32a0c74b527451fb723e3851d5d055b P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:16.956892 11928 raft_consensus.cc:2726] T 57ae3d10846a4aaa84eba81b13aafd44 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:16.958520 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_71eefae0e0f5450683c2e81bfef89116 [id=c01083628fb9483597ac93372c7f407e]
I20260924 00:21:16.959815 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_e2da51f4110147169f1bc08241696fae [id=c02abe55b70f4e23bc4a9f21300f39d6]
I20260924 00:21:16.961560 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_f36e3cefa9f4476594a81b22cf2b3491 [id=c173c5c646ac4ce1bedb4d8fb1bf083c]
I20260924 00:21:16.965270 11913 consensus_queue.cc:1035] T 4c2a729a802e4abe9ab4f989a40198be P c8626e300d0644b0908797ad61bd7cef [LEADER]: Connected to new peer: Peer: permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:16.966408 11913 consensus_queue.cc:1035] T 98926dd3850d40938b67779d5fd9a1db P c8626e300d0644b0908797ad61bd7cef [LEADER]: Connected to new peer: Peer: permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:16.973356 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_3f26bb58c6374373a24eb69faf5d68c6 [id=c5d285065f184765b488817af0b5d87d]
I20260924 00:21:16.975157 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_294eebbf563b4b24a4f75064b4e9233b [id=c98b19807d814947b152c5faba758be1]
I20260924 00:21:16.976590 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_b7b4d0037f4946659915a6bd5d65b202 [id=ca942906de0f4019bbdf72ec886fe6fb]
I20260924 00:21:16.981587 11039 tablet_bootstrap.cc:492] T 0de205104d7c4caf8f8f228e6a0d8c3a P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:16.981613 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_eff603db4cdf417c871be7eb94681227 [id=ccd80429571145e0bcb1ade6e3a315d4]
I20260924 00:21:16.981983 11039 ts_tablet_manager.cc:1397] T 0de205104d7c4caf8f8f228e6a0d8c3a P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.044s user 0.004s sys 0.006s
I20260924 00:21:16.982132 11918 ts_tablet_manager.cc:1918] T 57ae3d10846a4aaa84eba81b13aafd44 P c8626e300d0644b0908797ad61bd7cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:16.982597 11918 log.cc:1196] T 57ae3d10846a4aaa84eba81b13aafd44 P c8626e300d0644b0908797ad61bd7cef: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/57ae3d10846a4aaa84eba81b13aafd44
I20260924 00:21:16.982933 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_dbe5555ea7824aabb73edf7e94b89b03 [id=cceeb868b61c47e7a39d907e4f88000f]
I20260924 00:21:16.983397 11918 ts_tablet_manager.cc:1939] T 57ae3d10846a4aaa84eba81b13aafd44 P c8626e300d0644b0908797ad61bd7cef: Deleting consensus metadata
I20260924 00:21:16.984122 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_411ee5e9bc264adab9272f87158db777 [id=cd593507c8454e47998e55b67c95d11d]
I20260924 00:21:16.993981 10652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d32a0c74b527451fb723e3851d5d055b" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:21:16.994617 10652 raft_consensus.cc:3037] T d32a0c74b527451fb723e3851d5d055b P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:16.997447 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_acfad28894184501be9e3f9e38fea952 [id=cdcf0facca9849d990e8e9463f288410]
I20260924 00:21:16.998747 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_54a305f538ab4d24811b19d3e44e4a90 [id=d1071050336d4b9794f0ce1034da910f]
I20260924 00:21:16.999996 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_9194ac2d0ed7431385925285b2e7fc5d [id=d23bf403ee7d4863bd1e3d6ab879891a]
I20260924 00:21:17.002377 10652 raft_consensus.cc:2454] T d32a0c74b527451fb723e3851d5d055b P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 1.
I20260924 00:21:17.002569 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_e110cb436d444a319359e45630f8e2bb [id=d5ce3c6422db4e42912aabb3fd6c0e63]
I20260924 00:21:17.003894 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_c4f41dcffe2a4074b5178b60ec8e0884 [id=d7d5340352b948e3a72b210b1e99cc25]
I20260924 00:21:16.964057 11889 raft_consensus.cc:482] T 98926dd3850d40938b67779d5fd9a1db P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:17.005191 11889 raft_consensus.cc:504] T 98926dd3850d40938b67779d5fd9a1db P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:17.005745 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_5d4b70d8740f435bb43cb05808ec557e [id=dccf8774891e46649570c04d4ad36e6b]
I20260924 00:21:17.006912 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_4288ed69ffe84f6babc2d310dd022107 [id=de75a360e1ad4b50bc7e55815ca56f20]
I20260924 00:21:17.007453 11889 leader_election.cc:288] T 98926dd3850d40938b67779d5fd9a1db P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:17.008123 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_9670783a92074b639ecb0d6c374811e5 [id=df2a5a74db65431c9ecb750184f64709]
I20260924 00:21:17.009317 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_9e32aedf83114328ab54bc931f2a51a3 [id=e2acb1d9099a48798111f605b72f4d53]
I20260924 00:21:16.964980 11716 raft_consensus.cc:2258] T 717759eb61674c89b05b8d2b0dc34979 P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:17.011745 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_9680b062da604b83ad147f325a62ee9c [id=e7bb34d2413b49fcb6972b4f264dd774]
I20260924 00:21:17.018594 11716 ts_tablet_manager.cc:1905] T 717759eb61674c89b05b8d2b0dc34979 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:16.986646 11040 tablet_bootstrap.cc:492] T 5107b763acf04df88553ba0af2757570 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:17.025244 11040 ts_tablet_manager.cc:1397] T 5107b763acf04df88553ba0af2757570 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.191s user 0.009s sys 0.000s
I20260924 00:21:17.027483 11040 raft_consensus.cc:348] T 5107b763acf04df88553ba0af2757570 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:17.030570 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_06b78e500bd24799a8632eac4b4b108c [id=ed8b9a9a78d7481993cd6acca559d520]
I20260924 00:21:17.032471 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_a9fb0b9afed748b38640e91ac8eeedd2 [id=ee34e2e0b3564d109c4737f6093db745]
I20260924 00:21:16.993593 11041 tablet_bootstrap.cc:654] T 0de205104d7c4caf8f8f228e6a0d8c3a P b10a9082a37b46f09d581944914c1670: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:17.009325 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98926dd3850d40938b67779d5fd9a1db" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:17.033660 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_9b48f9449d9149128f9d014b582394ac [id=f2dc666553b843528f09119e75874b1e]
I20260924 00:21:17.039924 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_d50cb52a8cc04b5b8925615d0c9af895 [id=f45aef19d67844c3b266bc5ae753c480]
I20260924 00:21:17.041168 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_0a635e42c38a4b84a1ac9d8838333c9a [id=f6322db8cfe84fcfbaa47bb7a17e492e]
I20260924 00:21:16.983973 11039 raft_consensus.cc:348] T 0de205104d7c4caf8f8f228e6a0d8c3a P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:17.049137 11039 raft_consensus.cc:374] T 0de205104d7c4caf8f8f228e6a0d8c3a P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:17.049413 11039 raft_consensus.cc:729] T 0de205104d7c4caf8f8f228e6a0d8c3a P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:17.049978 11039 consensus_queue.cc:260] T 0de205104d7c4caf8f8f228e6a0d8c3a P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:16.993820 11799 leader_election.cc:288] T d32a0c74b527451fb723e3851d5d055b P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:17.010603 10939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98926dd3850d40938b67779d5fd9a1db" candidate_uuid: "48fbd603abe146c585d15196e1b5be08" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef" is_pre_election: true
I20260924 00:21:17.032346 10876 leader_election.cc:302] T d32a0c74b527451fb723e3851d5d055b P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08
I20260924 00:21:17.051864 11039 ts_tablet_manager.cc:1428] T 0de205104d7c4caf8f8f228e6a0d8c3a P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.070s user 0.002s sys 0.001s
I20260924 00:21:17.051959 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_b7a5eb7bd9fe4e0483e2d47d6182e46d [id=f79ca8d11aa04d5f8f72d58b798ed4ea]
I20260924 00:21:17.052598 11039 tablet_bootstrap.cc:492] T 0c5e5e375e414cd8b268e33b0c13b830 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:17.054054 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_4de2613c5f7243c2bb734ecd585ab408 [id=f8b875a3da5d406b9c8f0e553a5654fb]
I20260924 00:21:17.056682 11799 raft_consensus.cc:2781] T d32a0c74b527451fb723e3851d5d055b P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:17.057202 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_f0b5a94e5ae7437383840e76682bd5fa [id=ff0acca1a5c8440ca9e94f253f63df53]
I20260924 00:21:17.057282 11799 raft_consensus.cc:686] T d32a0c74b527451fb723e3851d5d055b P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Becoming Leader. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Running, Role: LEADER
I20260924 00:21:17.058434 11888 catalog_manager.cc:669] Loaded metadata for table default.Table_efa7000cc59a43639d964cd555a08961 [id=ff77e539ecb44767bb76cf076799a4c2]
I20260924 00:21:17.058357 11799 consensus_queue.cc:237] T d32a0c74b527451fb723e3851d5d055b P c8626e300d0644b0908797ad61bd7cef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:17.057526 11039 tablet_bootstrap.cc:654] T 0c5e5e375e414cd8b268e33b0c13b830 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:17.063344 11041 tablet_bootstrap.cc:492] T 0de205104d7c4caf8f8f228e6a0d8c3a P b10a9082a37b46f09d581944914c1670: No bootstrap required, opened a new log
I20260924 00:21:17.064378 11041 ts_tablet_manager.cc:1397] T 0de205104d7c4caf8f8f228e6a0d8c3a P b10a9082a37b46f09d581944914c1670: Time spent bootstrapping tablet: real 0.169s user 0.008s sys 0.003s
I20260924 00:21:17.066324 11041 raft_consensus.cc:348] T 0de205104d7c4caf8f8f228e6a0d8c3a P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:17.066929 11041 raft_consensus.cc:374] T 0de205104d7c4caf8f8f228e6a0d8c3a P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:17.067157 11041 raft_consensus.cc:729] T 0de205104d7c4caf8f8f228e6a0d8c3a P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b10a9082a37b46f09d581944914c1670, State: Initialized, Role: FOLLOWER
I20260924 00:21:17.075066 11039 tablet_bootstrap.cc:492] T 0c5e5e375e414cd8b268e33b0c13b830 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:17.075453 11039 ts_tablet_manager.cc:1397] T 0c5e5e375e414cd8b268e33b0c13b830 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.023s user 0.007s sys 0.002s
I20260924 00:21:17.077482 11039 raft_consensus.cc:348] T 0c5e5e375e414cd8b268e33b0c13b830 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:17.078119 11039 raft_consensus.cc:374] T 0c5e5e375e414cd8b268e33b0c13b830 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:16.999099 11854 raft_consensus.cc:504] T c7cb7c6892404c7fa593cdaaac67f800 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:17.078359 11039 raft_consensus.cc:729] T 0c5e5e375e414cd8b268e33b0c13b830 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
W20260924 00:21:17.080891 10876 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:33461, user_credentials={real_user=slave}} blocked reactor thread for 48800.7us
I20260924 00:21:17.081012 10797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7cb7c6892404c7fa593cdaaac67f800" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
I20260924 00:21:17.081907 10732 leader_election.cc:302] T 98926dd3850d40938b67779d5fd9a1db P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48fbd603abe146c585d15196e1b5be08; no voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef
W20260924 00:21:17.082957 10592 leader_election.cc:341] T c7cb7c6892404c7fa593cdaaac67f800 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:17.084715 10939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7cb7c6892404c7fa593cdaaac67f800" candidate_uuid: "b10a9082a37b46f09d581944914c1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8626e300d0644b0908797ad61bd7cef"
I20260924 00:21:17.079106 11039 consensus_queue.cc:260] T 0c5e5e375e414cd8b268e33b0c13b830 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:17.085253 10939 raft_consensus.cc:3037] T c7cb7c6892404c7fa593cdaaac67f800 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:17.086548 11039 ts_tablet_manager.cc:1428] T 0c5e5e375e414cd8b268e33b0c13b830 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20260924 00:21:17.087368 11039 tablet_bootstrap.cc:492] T 8e628abffd9545b391c25f83c9903732 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:17.091825 10939 raft_consensus.cc:2454] T c7cb7c6892404c7fa593cdaaac67f800 P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b10a9082a37b46f09d581944914c1670 in term 1.
I20260924 00:21:17.092932 10589 leader_election.cc:302] T c7cb7c6892404c7fa593cdaaac67f800 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08
I20260924 00:21:17.093667 11892 raft_consensus.cc:2781] T c7cb7c6892404c7fa593cdaaac67f800 P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:17.094067 11892 raft_consensus.cc:686] T c7cb7c6892404c7fa593cdaaac67f800 P b10a9082a37b46f09d581944914c1670 [term 1 LEADER]: Becoming Leader. State: Replica: b10a9082a37b46f09d581944914c1670, State: Running, Role: LEADER
I20260924 00:21:17.094763 11892 consensus_queue.cc:237] T c7cb7c6892404c7fa593cdaaac67f800 P b10a9082a37b46f09d581944914c1670 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:17.112548 11041 consensus_queue.cc:260] T 0de205104d7c4caf8f8f228e6a0d8c3a P b10a9082a37b46f09d581944914c1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:17.028188 11040 raft_consensus.cc:374] T 5107b763acf04df88553ba0af2757570 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:17.114105 11040 raft_consensus.cc:729] T 5107b763acf04df88553ba0af2757570 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:17.114928 11039 tablet_bootstrap.cc:654] T 8e628abffd9545b391c25f83c9903732 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:17.117484 11040 consensus_queue.cc:260] T 5107b763acf04df88553ba0af2757570 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:17.121140 11854 leader_election.cc:288] T c7cb7c6892404c7fa593cdaaac67f800 P b10a9082a37b46f09d581944914c1670 [CANDIDATE]: Term 1 election: Requested vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301)
I20260924 00:21:17.125054 10358 catalog_manager.cc:4841] TS c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): tablet 57ae3d10846a4aaa84eba81b13aafd44 (table default.Table_29e523da31634984b775f31084e82c99 [id=6d2a4bce1272466c8a00217643c0c9f5]) successfully deleted
I20260924 00:21:17.133886 11928 raft_consensus.cc:2726] T 98926dd3850d40938b67779d5fd9a1db P 48fbd603abe146c585d15196e1b5be08 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 00:21:17.152674 11040 ts_tablet_manager.cc:1428] T 5107b763acf04df88553ba0af2757570 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.127s user 0.005s sys 0.000s
I20260924 00:21:17.158543 11888 tablet_loader.cc:96] loaded metadata for tablet 004b12724c404a6fbedfd03fe6da2b2b (table default.Table_e33891e1d1b745819f1114e54bded8bb [id=b382a8277aa3452990df0abe27a80cb2])
I20260924 00:21:17.160190 11888 tablet_loader.cc:96] loaded metadata for tablet 05ccb3c5157c4a91b45c70e04927c213 (table default.Table_36c8c183203d418c9a15405725842b64 [id=3f954b8033264f41b0605ee408d21ed0])
I20260924 00:21:17.161248 11888 tablet_loader.cc:96] loaded metadata for tablet 0856ff2896ce4832af348947f226ee7b (table default.Table_9c2aa717e62340c785be379c0dd5837c [id=737634d9f0f4413c8b6dcaa26de87e2d])
I20260924 00:21:17.170536 11888 tablet_loader.cc:96] loaded metadata for tablet 08929edc5c5b4ecd80cef4fc472a9ffa (table default.Table_e735211dc38c444da1b3af42f354c740 [id=8031d25aceec4ca48e8b29e7d75e5320])
W20260924 00:21:17.171912 10360 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 83d67392af814e43a0fa1bee970820b8 (table default.Table_b4a6ce0bb6394e0aadb54d15db1f4b80 [id=02df80a9de5947dbbf8484189637255b], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 83d67392af814e43a0fa1bee970820b8 consensus state has no leader
I20260924 00:21:17.174093 11888 tablet_loader.cc:96] loaded metadata for tablet 0beb7ec3095549219734befe57d9daec (table default.Table_ceb867d167eb4df8bb2653ede8e37644 [id=b8ac1418c7d644b7b635d6cb28c100b6])
I20260924 00:21:17.175771 11888 tablet_loader.cc:96] loaded metadata for tablet 0c5e5e375e414cd8b268e33b0c13b830 (table default.Table_d50cb52a8cc04b5b8925615d0c9af895 [id=f45aef19d67844c3b266bc5ae753c480])
W20260924 00:21:17.187803 10592 consensus_peers.cc:487] T 1d5a76bc420b41098d4ebd3dc59aa1fc P b10a9082a37b46f09d581944914c1670 -> Peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Couldn't send request to peer 48fbd603abe146c585d15196e1b5be08. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260924 00:21:17.115376 11041 ts_tablet_manager.cc:1428] T 0de205104d7c4caf8f8f228e6a0d8c3a P b10a9082a37b46f09d581944914c1670: Time spent starting tablet: real 0.051s user 0.005s sys 0.000s
I20260924 00:21:17.199174 11041 tablet_bootstrap.cc:492] T d574642ebc6340fcb5e83c36804ee9f8 P b10a9082a37b46f09d581944914c1670: Bootstrap starting.
I20260924 00:21:17.207185 11039 tablet_bootstrap.cc:492] T 8e628abffd9545b391c25f83c9903732 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:17.207587 11039 ts_tablet_manager.cc:1397] T 8e628abffd9545b391c25f83c9903732 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.120s user 0.011s sys 0.018s
I20260924 00:21:17.176788 11888 tablet_loader.cc:96] loaded metadata for tablet 0c7b7a63ed8f44abb9872d36f756114d (table default.Table_6453acb2750442d0a172188dbc89d856 [id=0dd7e590773f4405bfb9c2a5060b3413])
I20260924 00:21:17.210765 11888 tablet_loader.cc:96] loaded metadata for tablet 0d555f059f84478eb805f4ca4534dab4 (table default.Table_99da18207fc542fbb68972724c4d410e [id=f75c1da3bd734e92bc105c473236fa9b])
I20260924 00:21:17.210798 11799 raft_consensus.cc:482] T 0c5e5e375e414cd8b268e33b0c13b830 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:17.211341 11799 raft_consensus.cc:504] T 0c5e5e375e414cd8b268e33b0c13b830 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:17.212527 11888 tablet_loader.cc:96] loaded metadata for tablet 0de944f51bc944459ceb31b254a38429 (table default.Table_4288ed69ffe84f6babc2d310dd022107 [id=de75a360e1ad4b50bc7e55815ca56f20])
I20260924 00:21:17.213459 11888 tablet_loader.cc:96] loaded metadata for tablet 0f3b1d46f2f640cbb9561bc5d214d223 (table default.Table_b918cc5bd6a94a2a91b42dc02bb68be3 [id=09c0270c40904e3089ed7f4a0609bab6])
I20260924 00:21:17.214357 11799 leader_election.cc:288] T 0c5e5e375e414cd8b268e33b0c13b830 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:17.214454 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c5e5e375e414cd8b268e33b0c13b830" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:17.214843 11799 raft_consensus.cc:482] T a53f06276dbb4d9f97a154a8ceb0278d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:17.215180 11799 raft_consensus.cc:504] T a53f06276dbb4d9f97a154a8ceb0278d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:17.215962 10652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c5e5e375e414cd8b268e33b0c13b830" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:17.216501 11039 raft_consensus.cc:348] T 8e628abffd9545b391c25f83c9903732 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:17.217119 11039 raft_consensus.cc:374] T 8e628abffd9545b391c25f83c9903732 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260924 00:21:17.217219 10878 leader_election.cc:341] T 0c5e5e375e414cd8b268e33b0c13b830 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:17.217360 11039 raft_consensus.cc:729] T 8e628abffd9545b391c25f83c9903732 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:17.217864 10654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a53f06276dbb4d9f97a154a8ceb0278d" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670" is_pre_election: true
I20260924 00:21:17.217941 11039 consensus_queue.cc:260] T 8e628abffd9545b391c25f83c9903732 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:17.218366 10654 raft_consensus.cc:2454] T a53f06276dbb4d9f97a154a8ceb0278d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 0.
W20260924 00:21:17.218638 10876 leader_election.cc:341] T 0c5e5e375e414cd8b268e33b0c13b830 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:17.218984 10876 leader_election.cc:302] T 0c5e5e375e414cd8b268e33b0c13b830 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670
I20260924 00:21:17.219731 11039 ts_tablet_manager.cc:1428] T 8e628abffd9545b391c25f83c9903732 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.012s user 0.005s sys 0.000s
I20260924 00:21:17.219875 11939 raft_consensus.cc:2726] T 0c5e5e375e414cd8b268e33b0c13b830 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:17.222349 10876 leader_election.cc:302] T a53f06276dbb4d9f97a154a8ceb0278d P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters:
I20260924 00:21:17.222957 11808 raft_consensus.cc:2781] T a53f06276dbb4d9f97a154a8ceb0278d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:17.223222 11808 raft_consensus.cc:482] T a53f06276dbb4d9f97a154a8ceb0278d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:17.223505 11808 raft_consensus.cc:3037] T a53f06276dbb4d9f97a154a8ceb0278d P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:17.227041 11039 tablet_bootstrap.cc:492] T 84e73180fab74524b4d568a010dc7175 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:17.229000 11888 tablet_loader.cc:96] loaded metadata for tablet 119d83562ef546cab496cd7b44fecc15 (table default.Table_76ffb857cf034b6f80598edd90dad953 [id=5fb5e1e90c9440f385e7b04520bcba9c])
I20260924 00:21:17.230218 11888 tablet_loader.cc:96] loaded metadata for tablet 13310c1530604e5595bf14dca31242c1 (table default.Table_81b3a0208acd453c8a35957f47f25ff3 [id=14d056bc2e7849ebb6e6ca7966e11324])
I20260924 00:21:17.233744 11040 tablet_bootstrap.cc:492] T cd30f52583124543b5d616ef94b31831 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:17.234269 11913 consensus_queue.cc:1035] T f85ac5b0fd2348d88e1076b781424ab3 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Connected to new peer: Peer: permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:17.234974 11039 tablet_bootstrap.cc:654] T 84e73180fab74524b4d568a010dc7175 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:17.235985 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a53f06276dbb4d9f97a154a8ceb0278d" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:17.237196 10878 leader_election.cc:341] T a53f06276dbb4d9f97a154a8ceb0278d P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:17.237524 11888 tablet_loader.cc:96] loaded metadata for tablet 153bbc88d9444285a01300eb778213a3 (table default.Table_204d831d7f94465c8b58927b4988ef31 [id=1105b489035347609ca1fecc85d2fae3])
I20260924 00:21:17.243078 11888 tablet_loader.cc:96] loaded metadata for tablet 195b033091b543b58ee08417eae2c0e6 (table default.Table_b7a5eb7bd9fe4e0483e2d47d6182e46d [id=f79ca8d11aa04d5f8f72d58b798ed4ea])
I20260924 00:21:17.240223 11808 raft_consensus.cc:504] T a53f06276dbb4d9f97a154a8ceb0278d P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:17.245424 10652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a53f06276dbb4d9f97a154a8ceb0278d" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b10a9082a37b46f09d581944914c1670"
I20260924 00:21:17.246105 10652 raft_consensus.cc:3037] T a53f06276dbb4d9f97a154a8ceb0278d P b10a9082a37b46f09d581944914c1670 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:17.252202 11888 tablet_loader.cc:96] loaded metadata for tablet 1cc672cf88554772bc4470d6120db6f6 (table default.Table_f1ae1e8ac652481680491d0da3355484 [id=36d61d1504cf495a9643bad809812d43])
I20260924 00:21:17.253341 11888 tablet_loader.cc:96] loaded metadata for tablet 1d5a76bc420b41098d4ebd3dc59aa1fc (table default.Table_efa7000cc59a43639d964cd555a08961 [id=ff77e539ecb44767bb76cf076799a4c2])
I20260924 00:21:17.256177 11888 tablet_loader.cc:96] loaded metadata for tablet 1ec36d59b5114feea144bcb72794136c (table default.Table_66a5eb5613d743c1b96af8b5a252fd85 [id=4dbb68e0a32349ad8da2945db650584d])
I20260924 00:21:17.257220 11888 tablet_loader.cc:96] loaded metadata for tablet 21d8bbcdcbff4a1abbeb9dba1a62883e (table default.Table_aa256627fd99496e988fab956cde2057 [id=579f89b1e70f4dc2b2080868adfeea76])
I20260924 00:21:17.258060 11888 tablet_loader.cc:96] loaded metadata for tablet 22640bf1c2304d978e66c002d74aab51 (table default.Table_0a7f75ce29d74b7da36e39b072bffccc [id=332608cae68d4f51aff9ab21f3bb9e12])
I20260924 00:21:17.260257 11888 tablet_loader.cc:96] loaded metadata for tablet 28be3abcaff64a3d97fad866472b41be (table default.Table_a01f8ccbfb7a4cff9315ec54f8f2f50e [id=001b691754074a66b8bbc7fa0f95caff])
I20260924 00:21:17.261607 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a53f06276dbb4d9f97a154a8ceb0278d" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08"
W20260924 00:21:17.262957 10878 leader_election.cc:341] T a53f06276dbb4d9f97a154a8ceb0278d P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:17.271804 11039 tablet_bootstrap.cc:492] T 84e73180fab74524b4d568a010dc7175 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:17.272190 11039 ts_tablet_manager.cc:1397] T 84e73180fab74524b4d568a010dc7175 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.045s user 0.002s sys 0.007s
I20260924 00:21:17.272471 11808 leader_election.cc:288] T a53f06276dbb4d9f97a154a8ceb0278d P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Requested vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:21:17.276619 11888 tablet_loader.cc:96] loaded metadata for tablet 2e839fd66fed4472a0892baa9aa85c54 (table default.Table_e724566b223445b7b3988eeb3e811bcd [id=4b08b25314a64fdd8313975f514fb10a])
I20260924 00:21:17.274286 11039 raft_consensus.cc:348] T 84e73180fab74524b4d568a010dc7175 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:17.277992 11039 raft_consensus.cc:374] T 84e73180fab74524b4d568a010dc7175 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:17.278247 11039 raft_consensus.cc:729] T 84e73180fab74524b4d568a010dc7175 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:17.278772 11039 consensus_queue.cc:260] T 84e73180fab74524b4d568a010dc7175 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:17.280836 11039 ts_tablet_manager.cc:1428] T 84e73180fab74524b4d568a010dc7175 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20260924 00:21:17.281718 11039 tablet_bootstrap.cc:492] T 915e8072851c4d2b90f8af12b1dc58e0 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:17.286324 10652 raft_consensus.cc:2454] T a53f06276dbb4d9f97a154a8ceb0278d P b10a9082a37b46f09d581944914c1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8626e300d0644b0908797ad61bd7cef in term 1.
I20260924 00:21:17.286981 11039 tablet_bootstrap.cc:654] T 915e8072851c4d2b90f8af12b1dc58e0 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:17.287495 11719 ts_tablet_manager.cc:1918] T 85a0ad61b7cb4e11b30e84ddc4655b64 P b10a9082a37b46f09d581944914c1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260924 00:21:17.288008 10876 leader_election.cc:302] T a53f06276dbb4d9f97a154a8ceb0278d P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b10a9082a37b46f09d581944914c1670, c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08
I20260924 00:21:17.288993 11808 raft_consensus.cc:2781] T a53f06276dbb4d9f97a154a8ceb0278d P c8626e300d0644b0908797ad61bd7cef [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:17.289695 11888 tablet_loader.cc:96] loaded metadata for tablet 3308ded6ebd740bba69c1ed73b489d85 (table default.Table_f24d66ccb67d4a5fb0c34a09c63e089c [id=6a53cdb2501c4276938cdeae3919624a])
I20260924 00:21:17.289657 11808 raft_consensus.cc:686] T a53f06276dbb4d9f97a154a8ceb0278d P c8626e300d0644b0908797ad61bd7cef [term 1 LEADER]: Becoming Leader. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Running, Role: LEADER
I20260924 00:21:17.290454 11808 consensus_queue.cc:237] T a53f06276dbb4d9f97a154a8ceb0278d P c8626e300d0644b0908797ad61bd7cef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:17.291668 11888 tablet_loader.cc:96] loaded metadata for tablet 36b1bc216f964b898503712eda33eeb6 (table default.Table_eff603db4cdf417c871be7eb94681227 [id=ccd80429571145e0bcb1ade6e3a315d4])
W20260924 00:21:17.300659 10878 consensus_peers.cc:487] T f85ac5b0fd2348d88e1076b781424ab3 P c8626e300d0644b0908797ad61bd7cef -> Peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Couldn't send request to peer 48fbd603abe146c585d15196e1b5be08. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260924 00:21:17.302178 10257 master-stress-test.cc:580] Killed test thread 1/10
I20260924 00:21:17.302470 10257 master-stress-test.cc:578] Killing test thread 2/10
I20260924 00:21:17.302711 10257 master-stress-test.cc:580] Killed test thread 2/10
I20260924 00:21:17.302914 10257 master-stress-test.cc:578] Killing test thread 3/10
I20260924 00:21:17.303155 10257 master-stress-test.cc:580] Killed test thread 3/10
I20260924 00:21:17.303354 10257 master-stress-test.cc:578] Killing test thread 4/10
I20260924 00:21:17.305570 10257 master-stress-test.cc:580] Killed test thread 4/10
I20260924 00:21:17.305786 10257 master-stress-test.cc:578] Killing test thread 5/10
I20260924 00:21:17.306008 10257 master-stress-test.cc:580] Killed test thread 5/10
I20260924 00:21:17.306202 10257 master-stress-test.cc:578] Killing test thread 6/10
I20260924 00:21:17.306416 10257 master-stress-test.cc:580] Killed test thread 6/10
I20260924 00:21:17.306593 10257 master-stress-test.cc:578] Killing test thread 7/10
I20260924 00:21:17.307057 10257 master-stress-test.cc:580] Killed test thread 7/10
I20260924 00:21:17.307262 10257 master-stress-test.cc:578] Killing test thread 8/10
I20260924 00:21:17.307541 10257 master-stress-test.cc:580] Killed test thread 8/10
I20260924 00:21:17.307722 10257 master-stress-test.cc:578] Killing test thread 9/10
I20260924 00:21:17.308007 10257 master-stress-test.cc:580] Killed test thread 9/10
I20260924 00:21:17.308214 10257 master-stress-test.cc:578] Killing test thread 10/10
I20260924 00:21:17.308521 10257 master-stress-test.cc:580] Killed test thread 10/10
I20260924 00:21:17.308856 10257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu with pid 10541
I20260924 00:21:17.297587 11888 tablet_loader.cc:96] loaded metadata for tablet 38e5e521ebab457cb403b841785d4d16 (table default.Table_e15931b8ec344adebc1ba60fa4be6f9b [id=5d3eb8f4da1b4b69ac59d69af11e5e46])
I20260924 00:21:17.312057 11888 tablet_loader.cc:96] loaded metadata for tablet 395c077731d740aabefd2f5dd80a67f8 (table default.Table_8f274639bca846bf89f8770c7de95055 [id=adababd4f08043c89167f5c28e00e6f0])
I20260924 00:21:17.312712 11928 raft_consensus.cc:482] T 5107b763acf04df88553ba0af2757570 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:17.313182 11888 tablet_loader.cc:96] loaded metadata for tablet 39cdf419c8b24274bbad1fe4444dd73e (table default.Table_b4a6ce0bb6394e0aadb54d15db1f4b80 [id=02df80a9de5947dbbf8484189637255b])
I20260924 00:21:17.313135 11928 raft_consensus.cc:504] T 5107b763acf04df88553ba0af2757570 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:17.314208 11799 leader_election.cc:288] T a53f06276dbb4d9f97a154a8ceb0278d P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:21:17.314908 11888 tablet_loader.cc:96] loaded metadata for tablet 3c850d0474414d2486cd7bbd6568e936 (table default.Table_294eebbf563b4b24a4f75064b4e9233b [id=c98b19807d814947b152c5faba758be1])
I20260924 00:21:17.314908 11928 leader_election.cc:288] T 5107b763acf04df88553ba0af2757570 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:17.316488 11888 tablet_loader.cc:96] loaded metadata for tablet 3f119ab5b64a467c9cff3b94d85b7935 (table default.Table_71eefae0e0f5450683c2e81bfef89116 [id=c01083628fb9483597ac93372c7f407e])
W20260924 00:21:17.316684 10732 leader_election.cc:341] T 5107b763acf04df88553ba0af2757570 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Not found: Tablet not found: 5107b763acf04df88553ba0af2757570
I20260924 00:21:17.326035 11888 tablet_loader.cc:96] loaded metadata for tablet 40bfe924a639439d956a387fb6c5e324 (table default.Table_9eeed352b33c49649d52ad128cb67203 [id=0b05d4353e924845834050bc136e812a])
I20260924 00:21:17.329720 11040 tablet_bootstrap.cc:654] T cd30f52583124543b5d616ef94b31831 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:17.334074 11888 tablet_loader.cc:96] loaded metadata for tablet 41bdfd499c6b42a2a7516c0157efddbb (table default.Table_b5c9933d8e0c481290863b60a642e8e7 [id=036fc1e2f392468ca0e9ce72610f3620])
I20260924 00:21:17.335211 11888 tablet_loader.cc:96] loaded metadata for tablet 4298ee4a320b470d89d0d4897bb50d5e (table default.Table_aceb5020c4374a028638d9931d6665c9 [id=100796b0a2be4926b6fca423e2b164fa])
I20260924 00:21:17.340427 11039 tablet_bootstrap.cc:492] T 915e8072851c4d2b90f8af12b1dc58e0 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:17.340842 11039 ts_tablet_manager.cc:1397] T 915e8072851c4d2b90f8af12b1dc58e0 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.059s user 0.012s sys 0.000s
I20260924 00:21:17.344908 11888 tablet_loader.cc:96] loaded metadata for tablet 4536327589e24e369b41ed4c73b0ac36 (table default.Table_a85971df84464ddd9ca760e1f0e2ec3f [id=38ce54e1abbf493e85934dfb3232e881])
I20260924 00:21:17.346681 11888 tablet_loader.cc:96] loaded metadata for tablet 45ba52ea61364b988ce77b265f8c7a79 (table default.Table_a5a83435a7fb47ce8083022c0686b74a [id=a587a50b0c204843a41acacc45c2b150])
I20260924 00:21:17.347590 11888 tablet_loader.cc:96] loaded metadata for tablet 46558e27033a485fbea9adebe2f0bebc (table default.Table_411ee5e9bc264adab9272f87158db777 [id=cd593507c8454e47998e55b67c95d11d])
I20260924 00:21:17.348475 11888 tablet_loader.cc:96] loaded metadata for tablet 4666b9ff4eaf47b1bca6977d28f13cb2 (table default.Table_396157d80ec44f329a275ccd3167bcb7 [id=4a372ce51c6b4915946217dec64719fa])
I20260924 00:21:17.349529 11888 tablet_loader.cc:96] loaded metadata for tablet 46f871141e464e4e994cdb8ee85e9e85 (table default.Table_3245e93d2b494073b531fd16a48f1244 [id=a33bb157c42440158194f77c2f9c07a7])
I20260924 00:21:17.350585 11888 tablet_loader.cc:96] loaded metadata for tablet 497c7d0c6f1b4a07bbc746f3cab1303d (table default.Table_4c92eecf56284783b041439a866855b8 [id=0311178133ae4be895516123aa4ea4fd])
I20260924 00:21:17.343034 11039 raft_consensus.cc:348] T 915e8072851c4d2b90f8af12b1dc58e0 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:17.351517 11716 ts_tablet_manager.cc:1918] T 717759eb61674c89b05b8d2b0dc34979 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:17.351600 11039 raft_consensus.cc:374] T 915e8072851c4d2b90f8af12b1dc58e0 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:17.351855 11039 raft_consensus.cc:729] T 915e8072851c4d2b90f8af12b1dc58e0 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:17.351969 11716 log.cc:1196] T 717759eb61674c89b05b8d2b0dc34979 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/717759eb61674c89b05b8d2b0dc34979
I20260924 00:21:17.352481 11039 consensus_queue.cc:260] T 915e8072851c4d2b90f8af12b1dc58e0 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:17.353497 11716 ts_tablet_manager.cc:1939] T 717759eb61674c89b05b8d2b0dc34979 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:17.354864 11039 ts_tablet_manager.cc:1428] T 915e8072851c4d2b90f8af12b1dc58e0 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.014s user 0.004s sys 0.000s
I20260924 00:21:17.365588 11039 tablet_bootstrap.cc:492] T b1c2eb7623f847a3a5b6964627ab94df P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:17.374271 11888 tablet_loader.cc:96] loaded metadata for tablet 4aa0dc5c502a406b99609f044b6baeb0 (table default.Table_82b79ff8ffca4a7880702b55cd10e393 [id=7e160144b1704e27a1d7f729c7a04049])
I20260924 00:21:17.374315 11040 tablet_bootstrap.cc:492] T cd30f52583124543b5d616ef94b31831 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:17.374907 11040 ts_tablet_manager.cc:1397] T cd30f52583124543b5d616ef94b31831 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.141s user 0.009s sys 0.000s
I20260924 00:21:17.377075 11040 raft_consensus.cc:348] T cd30f52583124543b5d616ef94b31831 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:17.377763 11040 raft_consensus.cc:374] T cd30f52583124543b5d616ef94b31831 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:17.378046 11040 raft_consensus.cc:729] T cd30f52583124543b5d616ef94b31831 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:17.378700 11040 consensus_queue.cc:260] T cd30f52583124543b5d616ef94b31831 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:17.380625 11040 ts_tablet_manager.cc:1428] T cd30f52583124543b5d616ef94b31831 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260924 00:21:17.385578 11888 tablet_loader.cc:96] loaded metadata for tablet 4bbc6df13fe14ecbae236bed68f037dc (table default.Table_e110cb436d444a319359e45630f8e2bb [id=d5ce3c6422db4e42912aabb3fd6c0e63])
I20260924 00:21:17.388057 11888 tablet_loader.cc:96] loaded metadata for tablet 4c2a729a802e4abe9ab4f989a40198be (table default.Table_5816c202c13e439cbd04eb58f456ebb5 [id=136a32ddaf2a4d2fbdddecf3ea95970c])
I20260924 00:21:17.389919 11888 tablet_loader.cc:96] loaded metadata for tablet 4e9f00abffe14046bf47a6cae7010690 (table default.Table_79486769678c44fe98623f5aa06975bb [id=423a3ef82b0f488fa8fb082d4e6f63a3])
I20260924 00:21:17.394131 11888 tablet_loader.cc:96] loaded metadata for tablet 5630f5ea96e64ed69e37d0d4bf9d9fd6 (table default.Table_a51a7905e6f848edb48e25b18d0ae196 [id=ba9dbd5eba46433d8166e91188c49599])
I20260924 00:21:17.395193 11888 tablet_loader.cc:96] loaded metadata for tablet 57a408ae6e4e439a852381a803bb8acf (table default.Table_99f14ba135cc459883dbb7c85dc21e30 [id=267c2c9545e547199df03ec06dda22be])
I20260924 00:21:17.398998 11888 tablet_loader.cc:96] loaded metadata for tablet 5b0345252c3c496db2987c30adce3da7 (table default.Table_a5d798f7819e403a96f27e98cb0af2bb [id=30a3f7c8fb9c46cf8b883eedae25d938])
I20260924 00:21:17.400046 11888 tablet_loader.cc:96] loaded metadata for tablet 5b43ce1fed0749b8b076b9b026914e07 (table default.Table_9608bad42525495eaa4df537b37969e7 [id=aa74d52589984aec8e6d296e5dffd623])
I20260924 00:21:17.401536 11039 tablet_bootstrap.cc:654] T b1c2eb7623f847a3a5b6964627ab94df P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:17.414813 11888 tablet_loader.cc:96] loaded metadata for tablet 5cb77a79c52f40b7b7b70366dcf6990a (table default.Table_69005ec88288442691befac5eff2adc7 [id=8308fedce3544a18912f343a3f4b519c])
I20260924 00:21:17.417433 11888 tablet_loader.cc:96] loaded metadata for tablet 5f2a32d5afff458fab79ffa106a98884 (table default.Table_bf61c7623556425688b99c1c373e4879 [id=4f70e833d7f74d91af0faa01ed52f78b])
I20260924 00:21:17.420536 11888 tablet_loader.cc:96] loaded metadata for tablet 61ca7994dac34681847fe5748509fe24 (table default.Table_a9fb0b9afed748b38640e91ac8eeedd2 [id=ee34e2e0b3564d109c4737f6093db745])
I20260924 00:21:17.421635 11040 tablet_bootstrap.cc:492] T d4e120acff3e4c30aed813b0b24fbfee P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:17.421598 11888 tablet_loader.cc:96] loaded metadata for tablet 6235db91988345f799160251990487d2 (table default.Table_5cf852dcccf3498f96d0e5e8364d1a84 [id=a15434697c7b4707a0effb20cbb36609])
I20260924 00:21:17.396150 11716 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35996 (request call id 347) took 2067 ms. Trace:
I20260924 00:21:17.422025 11716 rpcz_store.cc:276] 0924 00:21:15.329073 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:15.329208 (+ 135us) service_pool.cc:227] Handling call
0924 00:21:16.902144 (+1572936us) ts_tablet_manager.cc:1145] Deleting tablet 717759eb61674c89b05b8d2b0dc34979
0924 00:21:16.902185 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:17.124577 (+222392us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:17.351311 (+226734us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:17.396098 (+ 44787us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1572416}
I20260924 00:21:17.422489 11888 tablet_loader.cc:96] loaded metadata for tablet 62fc591e75274b9bae1bf12edb552ab8 (table default.Table_a9562c9df88a4874901902e8673278b5 [id=818963f97e3d4ae8a8e1d87e8511725d])
I20260924 00:21:17.423237 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet 717759eb61674c89b05b8d2b0dc34979 (table default.Table_9b991221b38645378eda8c1403ac0ce7 [id=b22dcf8c1c4346bebde8d0b9af8cf77c]) successfully deleted
I20260924 00:21:17.423488 11888 tablet_loader.cc:96] loaded metadata for tablet 63e9ebe18f824e18999c6aa5d8286f83 (table default.Table_ed94e40b06bf4340bd3a775e16a6d139 [id=0742097f4ab743959806cffe991bb5e5])
I20260924 00:21:17.424137 11716 tablet_replica.cc:330] T a62b85ebb2b84ae19cad3e3e2dd07a59 P 48fbd603abe146c585d15196e1b5be08: stopping tablet replica
I20260924 00:21:17.424834 11716 raft_consensus.cc:2229] T a62b85ebb2b84ae19cad3e3e2dd07a59 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:17.425203 11888 tablet_loader.cc:96] loaded metadata for tablet 667e84b0e2d1486289710ea36c594a66 (table default.Table_045585fb6067443fbd323a3bbc688220 [id=1c3a915542a1470785614e97b5779149])
I20260924 00:21:17.425341 11716 raft_consensus.cc:2258] T a62b85ebb2b84ae19cad3e3e2dd07a59 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:17.426949 11716 ts_tablet_manager.cc:1905] T a62b85ebb2b84ae19cad3e3e2dd07a59 P 48fbd603abe146c585d15196e1b5be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:17.437849 11888 tablet_loader.cc:96] loaded metadata for tablet 69061414550b4bd9bf4b2f40a38d8824 (table default.Table_c4f41dcffe2a4074b5178b60ec8e0884 [id=d7d5340352b948e3a72b210b1e99cc25])
I20260924 00:21:17.438798 11888 tablet_loader.cc:96] loaded metadata for tablet 6a163841c6d84daf92209057c9128d92 (table default.Table_e207dff5fb014842a20e58edf524849e [id=806be42cdd1b43f788520b28ecaf41a7])
I20260924 00:21:17.442515 11888 tablet_loader.cc:96] loaded metadata for tablet 6b8482823b4c40f0aedbf19d388d8013 (table default.Table_9e32aedf83114328ab54bc931f2a51a3 [id=e2acb1d9099a48798111f605b72f4d53])
I20260924 00:21:17.449275 11888 tablet_loader.cc:96] loaded metadata for tablet 6bcfd729e9bd4040b3384ab24164e80d (table default.Table_9670783a92074b639ecb0d6c374811e5 [id=df2a5a74db65431c9ecb750184f64709])
I20260924 00:21:17.449875 11928 raft_consensus.cc:482] T cd30f52583124543b5d616ef94b31831 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:17.450417 11888 tablet_loader.cc:96] loaded metadata for tablet 6d39cd33f352433eaf6857a78236aded (table default.Table_b65a4b1360e74260a30a4d0032e3c025 [id=244f1f1c84a14de9950e855b585177fe])
I20260924 00:21:17.450310 11928 raft_consensus.cc:504] T cd30f52583124543b5d616ef94b31831 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:17.451706 11888 tablet_loader.cc:96] loaded metadata for tablet 6f2195c79a9342bcbd1de472b56df83c (table default.Table_9b48f9449d9149128f9d014b582394ac [id=f2dc666553b843528f09119e75874b1e])
I20260924 00:21:17.451995 11928 leader_election.cc:288] T cd30f52583124543b5d616ef94b31831 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:17.452870 11888 tablet_loader.cc:96] loaded metadata for tablet 6fa654977f6442628aba1f478ea86358 (table default.Table_aa8bce70b4eb4f9a957369558580a1a2 [id=95fb80b2ff8044809780c83dcdd07cf0])
W20260924 00:21:17.454141 10732 leader_election.cc:341] T cd30f52583124543b5d616ef94b31831 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Not found: Tablet not found: cd30f52583124543b5d616ef94b31831
I20260924 00:21:17.458034 11888 tablet_loader.cc:96] loaded metadata for tablet 72d3bb80e1034881b94eb75de5d900c2 (table default.Table_b7b4d0037f4946659915a6bd5d65b202 [id=ca942906de0f4019bbdf72ec886fe6fb])
I20260924 00:21:17.459125 11888 tablet_loader.cc:96] loaded metadata for tablet 73018242c4f842e59160eb188ede78dc (table default.Table_acfad28894184501be9e3f9e38fea952 [id=cdcf0facca9849d990e8e9463f288410])
I20260924 00:21:17.460839 11888 tablet_loader.cc:96] loaded metadata for tablet 73bca39d6635475d83b38d11ce2a19d0 (table default.Table_d853fb40e0554242976678966283153e [id=3e6243f4f29c41db9fc70e43473ca92a])
I20260924 00:21:17.463220 11888 tablet_loader.cc:96] loaded metadata for tablet 7673597f1afc43198e86929a2939269a (table default.Table_b36ac15778564d608790f586457c6373 [id=93c1734de58245f8b1d5ab88ec9f0c8e])
I20260924 00:21:17.466831 11888 tablet_loader.cc:96] loaded metadata for tablet 83c1ce21578341ad8ee28ec44c518dc3 (table default.Table_a214c5e4b9584a1fba04c74e906fe4d3 [id=28864c6aa1f143bfb7b49d98bff16f8d])
I20260924 00:21:17.474381 11040 tablet_bootstrap.cc:654] T d4e120acff3e4c30aed813b0b24fbfee P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:17.483390 11888 tablet_loader.cc:96] loaded metadata for tablet 86d27e5be8614bef8fa05dc910ff3071 (table default.Table_c879109e72984beb85e10878ed02c18f [id=0912091b30374df895fec45a7eafe4cc])
I20260924 00:21:17.485051 11888 tablet_loader.cc:96] loaded metadata for tablet 8b768b600b434381acfa27528821e760 (table default.Table_ace1c8c32aaa43d6a39b2e3caba246be [id=5735e1d6c05e474e95239f5dc4b44a02])
I20260924 00:21:17.487473 11888 tablet_loader.cc:96] loaded metadata for tablet 8e99dc66ce9e41ef98b6b24fa41c1b36 (table default.Table_ee73693b7a6e46a3af53f7828da00f9e [id=9359b05533064c38b9bd589c0bb06471])
I20260924 00:21:17.487545 11039 tablet_bootstrap.cc:492] T b1c2eb7623f847a3a5b6964627ab94df P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:17.488026 11039 ts_tablet_manager.cc:1397] T b1c2eb7623f847a3a5b6964627ab94df P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.123s user 0.009s sys 0.000s
I20260924 00:21:17.488507 11888 tablet_loader.cc:96] loaded metadata for tablet 8fc17bdba09b40b7a3280c64121ff508 (table default.Table_bf986279c4534aa9bf4f06adb96f16ea [id=43419db539614db8802a84dff868f6b9])
I20260924 00:21:17.492158 11799 raft_consensus.cc:482] T d4e120acff3e4c30aed813b0b24fbfee P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:17.492581 11799 raft_consensus.cc:504] T d4e120acff3e4c30aed813b0b24fbfee P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:17.494297 11799 leader_election.cc:288] T d4e120acff3e4c30aed813b0b24fbfee P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:21:17.499686 11888 tablet_loader.cc:96] loaded metadata for tablet 94c87415a8ac42ada6b95e80b0b613b9 (table default.Table_b7b77b7460bf4db393567227b3536333 [id=6ca9fc0b6417483bacc49673da19e9c4])
I20260924 00:21:17.499673 11039 raft_consensus.cc:348] T b1c2eb7623f847a3a5b6964627ab94df P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:17.500420 11039 raft_consensus.cc:374] T b1c2eb7623f847a3a5b6964627ab94df P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:17.500687 11039 raft_consensus.cc:729] T b1c2eb7623f847a3a5b6964627ab94df P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:17.501343 11039 consensus_queue.cc:260] T b1c2eb7623f847a3a5b6964627ab94df P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:17.503463 11039 ts_tablet_manager.cc:1428] T b1c2eb7623f847a3a5b6964627ab94df P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.015s user 0.004s sys 0.000s
I20260924 00:21:17.497534 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4e120acff3e4c30aed813b0b24fbfee" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:17.521816 11888 tablet_loader.cc:96] loaded metadata for tablet 96960f3061fb43e9939b470f49edf1e9 (table default.Table_477c0f1cce9a4c089551e4e767ba7ae0 [id=6251651eb5944ee086dd564d86204148])
I20260924 00:21:17.523671 11888 tablet_loader.cc:96] loaded metadata for tablet 98926dd3850d40938b67779d5fd9a1db (table default.Table_22740cf31fbe40a58c31252f63417ad3 [id=74b4a09bdb464f27ae5d19a6a53ece5f])
I20260924 00:21:17.525409 11888 tablet_loader.cc:96] loaded metadata for tablet 9e44244181324288ad04afb7096839cb (table default.Table_dae0ffc8534047588f6f7cb13a610af5 [id=138be013fbcb4f01a6358d062200fe8f])
I20260924 00:21:17.529395 11888 tablet_loader.cc:96] loaded metadata for tablet a0668cc6cbc547c5bdc6b62a1b38fcde (table default.Table_16ca7c156bce492ca79eec1b9881fff6 [id=222d2010881c4b5c9635aa3c28adc72b])
I20260924 00:21:17.531769 11888 tablet_loader.cc:96] loaded metadata for tablet a24648494f8149b6993ccbc1665ab0d3 (table default.Table_6cebadb2663346f7b016f42a3ed0d1a5 [id=5b5b6b3fe58b433baf867711db148618])
I20260924 00:21:17.532858 11888 tablet_loader.cc:96] loaded metadata for tablet a3714466ae1441c2a9e32e26b9c37617 (table default.Table_2ff16b0fac094dc1865b40d6c91bbfdc [id=6fca68e9ecc44d71bae53204e1894542])
I20260924 00:21:17.533949 11888 tablet_loader.cc:96] loaded metadata for tablet a53491b1d40143caa73e802304436574 (table default.Table_eb75a8c5ce6e48b2ba6c7b79990467ef [id=21a378fad88643b79609e92dc1b96bff])
W20260924 00:21:17.508682 10878 leader_election.cc:341] T d4e120acff3e4c30aed813b0b24fbfee P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:17.535437 11039 tablet_bootstrap.cc:492] T 79d8dc7403f448df8cb960311eefe110 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:17.551503 11040 tablet_bootstrap.cc:492] T d4e120acff3e4c30aed813b0b24fbfee P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:17.551993 11040 ts_tablet_manager.cc:1397] T d4e120acff3e4c30aed813b0b24fbfee P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.131s user 0.008s sys 0.001s
I20260924 00:21:17.554136 11040 raft_consensus.cc:348] T d4e120acff3e4c30aed813b0b24fbfee P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:17.554544 11888 tablet_loader.cc:96] loaded metadata for tablet a5c560a0b5ce4a1ebb8c0011fe9217bf (table default.Table_7a350a3ca82c427298dd0bedb9db3d60 [id=775fb1f73d864bac8c0265786727798c])
I20260924 00:21:17.557181 11888 tablet_loader.cc:96] loaded metadata for tablet a6f54d92b6b047518389615308f0d57c (table default.Table_d1104bf9437443d48b9e769fcf06d5ec [id=5816a524b40d48629911e69babd7a80a])
I20260924 00:21:17.558925 11888 tablet_loader.cc:96] loaded metadata for tablet abcc79f486894131957e0895fec6d4f8 (table default.Table_d02a9cefdc804218a76ef33174772e09 [id=9037f3cc30d440c39da4db959a76b52f])
I20260924 00:21:17.560642 11888 tablet_loader.cc:96] loaded metadata for tablet ac8eb7c81dcb4fa099a10d511882c0d4 (table default.Table_4c16876881e6429fbe20b9eb73c68d3f [id=a672b0959c50464eb890db93fd96c64c])
I20260924 00:21:17.561065 11040 raft_consensus.cc:374] T d4e120acff3e4c30aed813b0b24fbfee P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:17.561403 11040 raft_consensus.cc:729] T d4e120acff3e4c30aed813b0b24fbfee P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:17.561760 11888 tablet_loader.cc:96] loaded metadata for tablet ad256de63e0948928a51d9231152ac2d (table default.Table_0a635e42c38a4b84a1ac9d8838333c9a [id=f6322db8cfe84fcfbaa47bb7a17e492e])
I20260924 00:21:17.562248 11040 consensus_queue.cc:260] T d4e120acff3e4c30aed813b0b24fbfee P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:17.567529 11039 tablet_bootstrap.cc:654] T 79d8dc7403f448df8cb960311eefe110 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:17.569342 11040 ts_tablet_manager.cc:1428] T d4e120acff3e4c30aed813b0b24fbfee P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.017s user 0.005s sys 0.000s
I20260924 00:21:17.555894 11716 ts_tablet_manager.cc:1918] T a62b85ebb2b84ae19cad3e3e2dd07a59 P 48fbd603abe146c585d15196e1b5be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:17.577877 11716 log.cc:1196] T a62b85ebb2b84ae19cad3e3e2dd07a59 P 48fbd603abe146c585d15196e1b5be08: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/a62b85ebb2b84ae19cad3e3e2dd07a59
I20260924 00:21:17.578691 11716 ts_tablet_manager.cc:1939] T a62b85ebb2b84ae19cad3e3e2dd07a59 P 48fbd603abe146c585d15196e1b5be08: Deleting consensus metadata
I20260924 00:21:17.579278 11888 tablet_loader.cc:96] loaded metadata for tablet afc8c856ce104cd9bfc4ba5d734c4fb9 (table default.Table_f36e3cefa9f4476594a81b22cf2b3491 [id=c173c5c646ac4ce1bedb4d8fb1bf083c])
I20260924 00:21:17.581015 11888 tablet_loader.cc:96] loaded metadata for tablet b0b3629be7c44bc4b3a79028e8dc33e9 (table default.Table_29e523da31634984b775f31084e82c99 [id=6d2a4bce1272466c8a00217643c0c9f5])
I20260924 00:21:17.582593 11888 tablet_loader.cc:96] loaded metadata for tablet b21185f093e54b4391ec8fe2891d6507 (table default.Table_16819990f8b0403a8947a30c882feb5a [id=bf8333796bff4b25908865364e81bfec])
I20260924 00:21:17.585086 11888 tablet_loader.cc:96] loaded metadata for tablet ba31383ba3a04edab4c250c4b3146185 (table default.Table_97629e88aeda47a880934c90cc33aaf6 [id=2613a8863ddb477498b493dfcc76114a])
I20260924 00:21:17.589893 11888 tablet_loader.cc:96] loaded metadata for tablet ba81ed510bca46fda6935905fa0c29fb (table default.Table_5d4b70d8740f435bb43cb05808ec557e [id=dccf8774891e46649570c04d4ad36e6b])
I20260924 00:21:17.591027 11888 tablet_loader.cc:96] loaded metadata for tablet bad371e91cb8460389a72e7beff7816d (table default.Table_8ff1c7565eff465da9ae77704e862894 [id=94267d66c5a74dd38e9f005f77b1ee18])
I20260924 00:21:17.592038 11888 tablet_loader.cc:96] loaded metadata for tablet bc23cb60e9e246ff8bc201a3087ffd4d (table default.Table_83435d6ddd64425390ab1b05337d2081 [id=bf5691bc3ec148eda2cb116bb7842cfc])
I20260924 00:21:17.593289 11799 raft_consensus.cc:482] T d574642ebc6340fcb5e83c36804ee9f8 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:17.593735 11799 raft_consensus.cc:504] T d574642ebc6340fcb5e83c36804ee9f8 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:17.595476 11040 tablet_bootstrap.cc:492] T 2e839fd66fed4472a0892baa9aa85c54 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:17.595582 11799 leader_election.cc:288] T d574642ebc6340fcb5e83c36804ee9f8 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:21:17.596907 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d574642ebc6340fcb5e83c36804ee9f8" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
W20260924 00:21:17.598668 10878 leader_election.cc:341] T d574642ebc6340fcb5e83c36804ee9f8 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:17.601226 11716 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35996 (request call id 348) took 1835 ms. Trace:
I20260924 00:21:17.601576 11716 rpcz_store.cc:276] 0924 00:21:15.765618 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:15.765727 (+ 109us) service_pool.cc:227] Handling call
0924 00:21:17.424001 (+1658274us) ts_tablet_manager.cc:1145] Deleting tablet a62b85ebb2b84ae19cad3e3e2dd07a59
0924 00:21:17.424042 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0924 00:21:17.511290 (+ 87248us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:17.555696 (+ 44406us) tablet_metadata.cc:641] Metadata flushed
0924 00:21:17.601171 (+ 45475us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1657759}
I20260924 00:21:17.602641 10361 catalog_manager.cc:4841] TS 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): tablet a62b85ebb2b84ae19cad3e3e2dd07a59 (table default.Table_3e21dcff4f0b40299e8ddb8ebb7f3c22 [id=3f6c757389bb4213b584f3e5a9ac7ede]) successfully deleted
I20260924 00:21:17.606105 11888 tablet_loader.cc:96] loaded metadata for tablet bc89341cd96e4c498160fb1412715eaa (table default.Table_6ba8cc42d97e4576a8c2021b39b78dbb [id=6ac68d6b800747d59768d4988137dae7])
I20260924 00:21:17.608776 11888 tablet_loader.cc:96] loaded metadata for tablet bf144ab1a6464a4ab02ac0642c5255e3 (table default.Table_382099cf77a24bb8bd6fe084926431c9 [id=3db896d304514ead8dfe726c402a6fcd])
I20260924 00:21:17.609982 11888 tablet_loader.cc:96] loaded metadata for tablet c03aa0298c0c4fe580c73cfc61f971a3 (table default.Table_54a305f538ab4d24811b19d3e44e4a90 [id=d1071050336d4b9794f0ce1034da910f])
I20260924 00:21:17.611034 11888 tablet_loader.cc:96] loaded metadata for tablet c06d4c830cca42e298a2c130fd8d76a8 (table default.Table_d775e9e0f1614f9fbbae3dfd8554f4c2 [id=2e31f472d7504629b66c629cb0125d24])
I20260924 00:21:17.611960 11040 tablet_bootstrap.cc:654] T 2e839fd66fed4472a0892baa9aa85c54 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:17.621065 11888 tablet_loader.cc:96] loaded metadata for tablet c1784f15ef0e4aaeb487f1ea9e52e83e (table default.Table_63927aa1872e41da917d6b92c8acb310 [id=485c42c942b74abbb32bea36d25bc184])
I20260924 00:21:17.622759 11799 raft_consensus.cc:482] T 84e73180fab74524b4d568a010dc7175 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:17.623145 11799 raft_consensus.cc:504] T 84e73180fab74524b4d568a010dc7175 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:17.624215 11039 tablet_bootstrap.cc:492] T 79d8dc7403f448df8cb960311eefe110 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:17.624576 11039 ts_tablet_manager.cc:1397] T 79d8dc7403f448df8cb960311eefe110 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.089s user 0.008s sys 0.000s
I20260924 00:21:17.625684 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84e73180fab74524b4d568a010dc7175" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:17.626564 11039 raft_consensus.cc:348] T 79d8dc7403f448df8cb960311eefe110 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
W20260924 00:21:17.627159 10878 leader_election.cc:341] T 84e73180fab74524b4d568a010dc7175 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:17.629050 11039 raft_consensus.cc:374] T 79d8dc7403f448df8cb960311eefe110 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:17.629321 11039 raft_consensus.cc:729] T 79d8dc7403f448df8cb960311eefe110 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:17.630082 11928 raft_consensus.cc:482] T 9319f9caadb04067a2e9700b5019ed37 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:17.629891 11039 consensus_queue.cc:260] T 79d8dc7403f448df8cb960311eefe110 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:17.630496 11928 raft_consensus.cc:504] T 9319f9caadb04067a2e9700b5019ed37 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:17.631247 11799 leader_election.cc:288] T 84e73180fab74524b4d568a010dc7175 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:17.632201 11928 leader_election.cc:288] T 9319f9caadb04067a2e9700b5019ed37 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
W20260924 00:21:17.633793 10732 leader_election.cc:341] T 9319f9caadb04067a2e9700b5019ed37 P 48fbd603abe146c585d15196e1b5be08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8626e300d0644b0908797ad61bd7cef (127.10.4.67:41301): Not found: Tablet not found: 9319f9caadb04067a2e9700b5019ed37
I20260924 00:21:17.648669 11039 ts_tablet_manager.cc:1428] T 79d8dc7403f448df8cb960311eefe110 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.024s user 0.004s sys 0.003s
I20260924 00:21:17.649706 11039 tablet_bootstrap.cc:492] T 1492ece2346d461ea57096a897daf83f P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:17.664474 11040 tablet_bootstrap.cc:492] T 2e839fd66fed4472a0892baa9aa85c54 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:17.664878 11040 ts_tablet_manager.cc:1397] T 2e839fd66fed4472a0892baa9aa85c54 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.070s user 0.008s sys 0.001s
I20260924 00:21:17.665905 11888 tablet_loader.cc:96] loaded metadata for tablet c467913f46b84705a1718372d90ad579 (table default.Table_35151b4e43bd4e3bbf312c8f1c619e91 [id=3bdd316445814324a47269d8249a2089])
I20260924 00:21:17.666913 11040 raft_consensus.cc:348] T 2e839fd66fed4472a0892baa9aa85c54 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:17.667527 11040 raft_consensus.cc:374] T 2e839fd66fed4472a0892baa9aa85c54 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:17.667908 11040 raft_consensus.cc:729] T 2e839fd66fed4472a0892baa9aa85c54 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:17.668576 11040 consensus_queue.cc:260] T 2e839fd66fed4472a0892baa9aa85c54 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:17.670053 11799 raft_consensus.cc:482] T 0de205104d7c4caf8f8f228e6a0d8c3a P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:17.670486 11799 raft_consensus.cc:504] T 0de205104d7c4caf8f8f228e6a0d8c3a P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:17.672215 11799 leader_election.cc:288] T 0de205104d7c4caf8f8f228e6a0d8c3a P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:17.680697 11888 tablet_loader.cc:96] loaded metadata for tablet c7ed17dee33d40d5b50c96340199048b (table default.Table_4de2613c5f7243c2bb734ecd585ab408 [id=f8b875a3da5d406b9c8f0e553a5654fb])
I20260924 00:21:17.683041 11039 tablet_bootstrap.cc:654] T 1492ece2346d461ea57096a897daf83f P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:17.673616 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0de205104d7c4caf8f8f228e6a0d8c3a" candidate_uuid: "c8626e300d0644b0908797ad61bd7cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fbd603abe146c585d15196e1b5be08" is_pre_election: true
I20260924 00:21:17.687832 11888 tablet_loader.cc:96] loaded metadata for tablet c889f258d75448faafa7bb65e2aafc12 (table default.Table_e06918102eb24c628d528f20522dd2a4 [id=0b40ac1ec523496badcb50a185abeaf6])
I20260924 00:21:17.688989 11888 tablet_loader.cc:96] loaded metadata for tablet ca51a5f7e0644df0a793a8be8dbc271e (table default.Table_cd15e5e8e07947459840c3ed5dca1f1d [id=ad71d06959604cdf89d7dbe5716d2d08])
I20260924 00:21:17.690145 11888 tablet_loader.cc:96] loaded metadata for tablet cac5464ea95543b1942c55bdf583db50 (table default.Table_f8be74da4e7043749c326a27f9bc4bdf [id=0b35a077ccef43beba6f4e048b1ca46e])
I20260924 00:21:17.693810 11888 tablet_loader.cc:96] loaded metadata for tablet cf0f0f016d074b1fb78d9b427c67fe86 (table default.Table_e412cbac77574c5aafc587eaa698b633 [id=667f4349af3f4eb28461e33f65beeb1c])
I20260924 00:21:17.695480 11888 tablet_loader.cc:96] loaded metadata for tablet cf7f64ac8821440c861ace6a476b55b6 (table default.Table_d90e8159c7b34d59bcde6044c2e70bb8 [id=526561ff42b043169b418f9996d8006b])
I20260924 00:21:17.698581 11888 tablet_loader.cc:96] loaded metadata for tablet d574642ebc6340fcb5e83c36804ee9f8 (table default.Table_9f2df24361da4f9a9340f6f8673043ea [id=59ec3e94e2f44150994a5ffc1d9fa8f4])
I20260924 00:21:17.700213 11888 tablet_loader.cc:96] loaded metadata for tablet d84590b70d83434fa50778a8ead79845 (table default.Table_b89f627e972b4c3e841bdb133667aa66 [id=2dd95b399ca84b569e46218f207f5950])
I20260924 00:21:17.700498 11040 ts_tablet_manager.cc:1428] T 2e839fd66fed4472a0892baa9aa85c54 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.035s user 0.004s sys 0.000s
I20260924 00:21:17.701270 11888 tablet_loader.cc:96] loaded metadata for tablet dec9e84232d74a2d8d5717cb432ab4d0 (table default.Table_9680b062da604b83ad147f325a62ee9c [id=e7bb34d2413b49fcb6972b4f264dd774])
I20260924 00:21:17.701327 11040 tablet_bootstrap.cc:492] T f85ac5b0fd2348d88e1076b781424ab3 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
I20260924 00:21:17.702342 11888 tablet_loader.cc:96] loaded metadata for tablet df03f899e5de4c12b16fbb40bdb1f33a (table default.Table_24047fc04a434959928e6148f187a672 [id=64a8cee150dd4ad4bba25adf7dbe36e5])
I20260924 00:21:17.705718 11888 tablet_loader.cc:96] loaded metadata for tablet e26b96e7c0ab4523b9a725743e2d7625 (table default.Table_14437b13dbe34d5591c2a180c12c23b1 [id=aa1298d4fa2d40ac93e7c74077f294ce])
I20260924 00:21:17.706799 11888 tablet_loader.cc:96] loaded metadata for tablet e33db14b4aec4bcd8c6f16dd418a23fe (table default.Table_cae9fb8a84b248ac96ee0c3ee9874324 [id=972f6bb2a2094f5a9a49ac19ff3e0f71])
I20260924 00:21:17.707855 11888 tablet_loader.cc:96] loaded metadata for tablet e3dbe5c4827344ed8fe19ae826f62709 (table default.Table_9e4907fea442455b9ac9c793dfabf496 [id=41a14bd521754a1e8fdd166c7a83bd62])
I20260924 00:21:17.709671 11888 tablet_loader.cc:96] loaded metadata for tablet e489ff048ced45d2b16112934d0bb3c1 (table default.Table_b07b221a38e14a93aa27e7c2dfe55549 [id=79232ff751084c1292ba92127d124c43])
I20260924 00:21:17.710724 11888 tablet_loader.cc:96] loaded metadata for tablet e68c8ce351bf4f699fddbc0689803ab8 (table default.Table_9194ac2d0ed7431385925285b2e7fc5d [id=d23bf403ee7d4863bd1e3d6ab879891a])
I20260924 00:21:17.711755 11888 tablet_loader.cc:96] loaded metadata for tablet e880856244d84c18bb09b13566130acb (table default.Table_cbd6d689c0a149d6a22cc163d3efb580 [id=3c20d5d6a1b3460882aa8f3f1bb71184])
W20260924 00:21:17.684314 10878 leader_election.cc:341] T 0de205104d7c4caf8f8f228e6a0d8c3a P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:17.713642 11888 tablet_loader.cc:96] loaded metadata for tablet e97db32bea1a4cf1bea82d99c8db8147 (table default.Table_8823ba2975054860816e34a4fa104be8 [id=556a0c70dd484c5aaa120eb5f0dfe8e1])
I20260924 00:21:17.714684 11888 tablet_loader.cc:96] loaded metadata for tablet ea7158417c944cbe9f46121964a87789 (table default.Table_dbe5555ea7824aabb73edf7e94b89b03 [id=cceeb868b61c47e7a39d907e4f88000f])
I20260924 00:21:17.716275 11888 tablet_loader.cc:96] loaded metadata for tablet eecf148fee6549388c33a66c124e9426 (table default.Table_d1738e7eeecb4601a9402b1745b33bd3 [id=8875bd95730045a4b0bfc9176f0c023e])
I20260924 00:21:17.717222 11888 tablet_loader.cc:96] loaded metadata for tablet f220c1fadaf046f6bf085fcdf87109dd (table default.Table_06b78e500bd24799a8632eac4b4b108c [id=ed8b9a9a78d7481993cd6acca559d520])
I20260924 00:21:17.720700 11888 tablet_loader.cc:96] loaded metadata for tablet f3abf4ab0f36450c9effcca01aff0719 (table default.Table_4541883eff524699a7226b51ad3b9727 [id=896bcf73282349db8fab4222fc76ca71])
I20260924 00:21:17.721836 11888 tablet_loader.cc:96] loaded metadata for tablet f4db89087d404354b4392344715b7e97 (table default.Table_5bdf50c2b69b4f3cadbb52a11cf10035 [id=b07edcb7ee5044c6a486984cc3efe917])
I20260924 00:21:17.722877 11888 tablet_loader.cc:96] loaded metadata for tablet f72cedef5fe94074a8233e657e9346c8 (table default.Table_f0b5a94e5ae7437383840e76682bd5fa [id=ff0acca1a5c8440ca9e94f253f63df53])
I20260924 00:21:17.727353 11040 tablet_bootstrap.cc:654] T f85ac5b0fd2348d88e1076b781424ab3 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:17.727319 11888 tablet_loader.cc:96] loaded metadata for tablet fade2106cef748c5adc2af770027df63 (table default.Table_e2da51f4110147169f1bc08241696fae [id=c02abe55b70f4e23bc4a9f21300f39d6])
I20260924 00:21:17.728844 11888 tablet_loader.cc:96] loaded metadata for tablet fcccd9816d0642b5b4a9fc2d4be59464 (table default.Table_f9897a81fc084a25ae405522cf5c7ef9 [id=7a6f8b0af8484d8b80c02e80027db9d4])
I20260924 00:21:17.731027 11888 tablet_loader.cc:96] loaded metadata for tablet fd0c3f766a464116aa777b849cf4acc2 (table default.Table_3f26bb58c6374373a24eb69faf5d68c6 [id=c5d285065f184765b488817af0b5d87d])
I20260924 00:21:17.733299 11888 tablet_loader.cc:96] loaded metadata for tablet fe43761fcbd6415c8c250a4c419f7aa2 (table default.Table_29d22b606c3a4d4ba2fce79bd410c4f4 [id=9839446621034266a1e4815b8f4863c2])
I20260924 00:21:17.737473 11039 tablet_bootstrap.cc:492] T 1492ece2346d461ea57096a897daf83f P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:17.737974 11039 ts_tablet_manager.cc:1397] T 1492ece2346d461ea57096a897daf83f P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.088s user 0.008s sys 0.001s
I20260924 00:21:17.739984 11039 raft_consensus.cc:348] T 1492ece2346d461ea57096a897daf83f P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
W20260924 00:21:17.740386 11888 catalog_manager.cc:1476] Time spent T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc: Loading table and tablet metadata into memory: real 1.820s user 0.439s sys 0.039s
I20260924 00:21:17.740705 11888 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:21:17.740671 11039 raft_consensus.cc:374] T 1492ece2346d461ea57096a897daf83f P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:17.741034 11039 raft_consensus.cc:729] T 1492ece2346d461ea57096a897daf83f P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:17.748085 10795 raft_consensus.cc:3037] T 2e839fd66fed4472a0892baa9aa85c54 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:17.756078 11799 consensus_queue.cc:1035] T 2e839fd66fed4472a0892baa9aa85c54 P c8626e300d0644b0908797ad61bd7cef [LEADER]: Connected to new peer: Peer: permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:17.766052 11040 tablet_bootstrap.cc:492] T f85ac5b0fd2348d88e1076b781424ab3 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:17.741698 11039 consensus_queue.cc:260] T 1492ece2346d461ea57096a897daf83f P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:17.769727 11039 ts_tablet_manager.cc:1428] T 1492ece2346d461ea57096a897daf83f P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.031s user 0.004s sys 0.000s
I20260924 00:21:17.770710 11040 ts_tablet_manager.cc:1397] T f85ac5b0fd2348d88e1076b781424ab3 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.070s user 0.009s sys 0.000s
I20260924 00:21:17.791422 11040 raft_consensus.cc:348] T f85ac5b0fd2348d88e1076b781424ab3 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:17.808493 11040 raft_consensus.cc:374] T f85ac5b0fd2348d88e1076b781424ab3 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:17.810910 11039 tablet_bootstrap.cc:492] T c03aa0298c0c4fe580c73cfc61f971a3 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:17.829208 11040 raft_consensus.cc:729] T f85ac5b0fd2348d88e1076b781424ab3 P 48fbd603abe146c585d15196e1b5be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fbd603abe146c585d15196e1b5be08, State: Initialized, Role: FOLLOWER
I20260924 00:21:17.830571 11040 consensus_queue.cc:260] T f85ac5b0fd2348d88e1076b781424ab3 P 48fbd603abe146c585d15196e1b5be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:17.851446 11040 ts_tablet_manager.cc:1428] T f85ac5b0fd2348d88e1076b781424ab3 P 48fbd603abe146c585d15196e1b5be08: Time spent starting tablet: real 0.077s user 0.004s sys 0.000s
I20260924 00:21:17.855527 11039 tablet_bootstrap.cc:654] T c03aa0298c0c4fe580c73cfc61f971a3 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:17.866637 11040 tablet_bootstrap.cc:492] T c7cb7c6892404c7fa593cdaaac67f800 P 48fbd603abe146c585d15196e1b5be08: Bootstrap starting.
W20260924 00:21:17.862040 10878 consensus_peers.cc:487] T d32a0c74b527451fb723e3851d5d055b P c8626e300d0644b0908797ad61bd7cef -> Peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Couldn't send request to peer 48fbd603abe146c585d15196e1b5be08. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260924 00:21:17.896761 11040 tablet_bootstrap.cc:654] T c7cb7c6892404c7fa593cdaaac67f800 P 48fbd603abe146c585d15196e1b5be08: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:17.901772 11039 tablet_bootstrap.cc:492] T c03aa0298c0c4fe580c73cfc61f971a3 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:17.902133 11039 ts_tablet_manager.cc:1397] T c03aa0298c0c4fe580c73cfc61f971a3 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.091s user 0.008s sys 0.000s
I20260924 00:21:17.904309 11039 raft_consensus.cc:348] T c03aa0298c0c4fe580c73cfc61f971a3 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:17.904904 11039 raft_consensus.cc:374] T c03aa0298c0c4fe580c73cfc61f971a3 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:17.905150 11039 raft_consensus.cc:729] T c03aa0298c0c4fe580c73cfc61f971a3 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:17.907452 11039 consensus_queue.cc:260] T c03aa0298c0c4fe580c73cfc61f971a3 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
W20260924 00:21:17.912287 10876 connection.cc:535] client connection to 127.10.4.65:33461 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260924 00:21:17.912711 10876 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260924 00:21:17.921643 11040 tablet_bootstrap.cc:492] T c7cb7c6892404c7fa593cdaaac67f800 P 48fbd603abe146c585d15196e1b5be08: No bootstrap required, opened a new log
I20260924 00:21:17.922788 11040 ts_tablet_manager.cc:1397] T c7cb7c6892404c7fa593cdaaac67f800 P 48fbd603abe146c585d15196e1b5be08: Time spent bootstrapping tablet: real 0.056s user 0.009s sys 0.000s
W20260924 00:21:17.923996 10359 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.4.65:33461 (error 108) [suppressed 2 similar messages]
W20260924 00:21:17.924810 10733 connection.cc:535] client connection to 127.10.4.65:33461 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260924 00:21:17.925081 11039 ts_tablet_manager.cc:1428] T c03aa0298c0c4fe580c73cfc61f971a3 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.023s user 0.006s sys 0.000s
W20260924 00:21:17.925319 10733 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260924 00:21:17.935707 11888 catalog_manager.cc:1259] Loaded cluster ID: 89ba9572944348c7affb30a26e8aad8d
I20260924 00:21:17.935981 11888 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:21:17.939041 11039 tablet_bootstrap.cc:492] T d71ce6053fdd4d53b3e0e596148d4c5c P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:17.949084 11039 tablet_bootstrap.cc:654] T d71ce6053fdd4d53b3e0e596148d4c5c P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:17.954160 10257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu with pid 10709
W20260924 00:21:17.964021 10359 catalog_manager.cc:4584] TS b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 85a0ad61b7cb4e11b30e84ddc4655b64: Network error: Client connection negotiation failed: client connection to 127.10.4.65:33461: connect: Connection refused (error 111)
I20260924 00:21:17.965565 11039 tablet_bootstrap.cc:492] T d71ce6053fdd4d53b3e0e596148d4c5c P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:17.965924 11039 ts_tablet_manager.cc:1397] T d71ce6053fdd4d53b3e0e596148d4c5c P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.027s user 0.011s sys 0.000s
I20260924 00:21:17.968051 11039 raft_consensus.cc:348] T d71ce6053fdd4d53b3e0e596148d4c5c P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:17.968667 11039 raft_consensus.cc:374] T d71ce6053fdd4d53b3e0e596148d4c5c P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:17.968901 11039 raft_consensus.cc:729] T d71ce6053fdd4d53b3e0e596148d4c5c P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:17.970069 11039 consensus_queue.cc:260] T d71ce6053fdd4d53b3e0e596148d4c5c P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
W20260924 00:21:17.975385 10876 consensus_peers.cc:487] T 119d83562ef546cab496cd7b44fecc15 P c8626e300d0644b0908797ad61bd7cef -> Peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Couldn't send request to peer b10a9082a37b46f09d581944914c1670. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:33461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 00:21:17.975777 10876 consensus_peers.cc:487] T 5f2a32d5afff458fab79ffa106a98884 P c8626e300d0644b0908797ad61bd7cef -> Peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Couldn't send request to peer b10a9082a37b46f09d581944914c1670. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:33461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 00:21:17.976897 10876 consensus_peers.cc:487] T 195b033091b543b58ee08417eae2c0e6 P c8626e300d0644b0908797ad61bd7cef -> Peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Couldn't send request to peer b10a9082a37b46f09d581944914c1670. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:33461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 00:21:17.977444 10876 consensus_peers.cc:487] T 72d3bb80e1034881b94eb75de5d900c2 P c8626e300d0644b0908797ad61bd7cef -> Peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Couldn't send request to peer b10a9082a37b46f09d581944914c1670. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:33461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 00:21:17.977743 10876 leader_election.cc:334] T d4e120acff3e4c30aed813b0b24fbfee P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Network error: Client connection negotiation failed: client connection to 127.10.4.65:33461: connect: Connection refused (error 111)
I20260924 00:21:17.978058 10876 leader_election.cc:302] T d4e120acff3e4c30aed813b0b24fbfee P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670
I20260924 00:21:17.985144 11939 raft_consensus.cc:2726] T d4e120acff3e4c30aed813b0b24fbfee P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260924 00:21:17.985960 10876 consensus_peers.cc:487] T 5b0345252c3c496db2987c30adce3da7 P c8626e300d0644b0908797ad61bd7cef -> Peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Couldn't send request to peer b10a9082a37b46f09d581944914c1670. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:33461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 00:21:17.986390 10876 leader_election.cc:334] T d574642ebc6340fcb5e83c36804ee9f8 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Network error: Client connection negotiation failed: client connection to 127.10.4.65:33461: connect: Connection refused (error 111)
I20260924 00:21:17.986743 10876 leader_election.cc:302] T d574642ebc6340fcb5e83c36804ee9f8 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670
I20260924 00:21:17.987345 11939 raft_consensus.cc:2726] T d574642ebc6340fcb5e83c36804ee9f8 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260924 00:21:17.988095 10876 consensus_peers.cc:487] T 2e839fd66fed4472a0892baa9aa85c54 P c8626e300d0644b0908797ad61bd7cef -> Peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Couldn't send request to peer b10a9082a37b46f09d581944914c1670. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:33461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 00:21:17.988551 10876 leader_election.cc:334] T 84e73180fab74524b4d568a010dc7175 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Network error: Client connection negotiation failed: client connection to 127.10.4.65:33461: connect: Connection refused (error 111)
I20260924 00:21:17.988898 10876 leader_election.cc:302] T 84e73180fab74524b4d568a010dc7175 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670
I20260924 00:21:17.990598 11939 raft_consensus.cc:2726] T 84e73180fab74524b4d568a010dc7175 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260924 00:21:17.991384 10876 leader_election.cc:334] T 0de205104d7c4caf8f8f228e6a0d8c3a P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Network error: Client connection negotiation failed: client connection to 127.10.4.65:33461: connect: Connection refused (error 111)
I20260924 00:21:17.994207 11039 ts_tablet_manager.cc:1428] T d71ce6053fdd4d53b3e0e596148d4c5c P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.028s user 0.006s sys 0.000s
I20260924 00:21:17.996196 10876 leader_election.cc:302] T 0de205104d7c4caf8f8f228e6a0d8c3a P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670
I20260924 00:21:17.996825 11039 tablet_bootstrap.cc:492] T 63e9ebe18f824e18999c6aa5d8286f83 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:17.996898 11939 raft_consensus.cc:2726] T 0de205104d7c4caf8f8f228e6a0d8c3a P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260924 00:21:17.997720 10876 consensus_peers.cc:487] T 98926dd3850d40938b67779d5fd9a1db P c8626e300d0644b0908797ad61bd7cef -> Peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Couldn't send request to peer b10a9082a37b46f09d581944914c1670. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:33461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 00:21:17.998124 10876 consensus_peers.cc:487] T 4c2a729a802e4abe9ab4f989a40198be P c8626e300d0644b0908797ad61bd7cef -> Peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Couldn't send request to peer b10a9082a37b46f09d581944914c1670. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:33461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 00:21:17.999743 11939 raft_consensus.cc:482] T 915e8072851c4d2b90f8af12b1dc58e0 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:18.000109 11939 raft_consensus.cc:504] T 915e8072851c4d2b90f8af12b1dc58e0 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } } peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } }
I20260924 00:21:18.001829 11939 leader_election.cc:288] T 915e8072851c4d2b90f8af12b1dc58e0 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461), 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359)
I20260924 00:21:18.013175 11039 tablet_bootstrap.cc:654] T 63e9ebe18f824e18999c6aa5d8286f83 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
W20260924 00:21:18.028873 10876 leader_election.cc:334] T 915e8072851c4d2b90f8af12b1dc58e0 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Network error: Client connection negotiation failed: client connection to 127.10.4.65:33461: connect: Connection refused (error 111)
I20260924 00:21:18.030416 11039 tablet_bootstrap.cc:492] T 63e9ebe18f824e18999c6aa5d8286f83 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:18.030855 11039 ts_tablet_manager.cc:1397] T 63e9ebe18f824e18999c6aa5d8286f83 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.034s user 0.009s sys 0.003s
I20260924 00:21:18.032971 11039 raft_consensus.cc:348] T 63e9ebe18f824e18999c6aa5d8286f83 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:18.033571 11039 raft_consensus.cc:374] T 63e9ebe18f824e18999c6aa5d8286f83 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:18.033834 11039 raft_consensus.cc:729] T 63e9ebe18f824e18999c6aa5d8286f83 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:18.041592 11039 consensus_queue.cc:260] T 63e9ebe18f824e18999c6aa5d8286f83 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
W20260924 00:21:18.049969 11961 log.cc:924] Time spent T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92: Append to log took a long time: real 0.136s user 0.012s sys 0.003s
I20260924 00:21:18.055883 11039 ts_tablet_manager.cc:1428] T 63e9ebe18f824e18999c6aa5d8286f83 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.025s user 0.005s sys 0.000s
I20260924 00:21:18.068383 11039 tablet_bootstrap.cc:492] T a6e4030e824047ca9cc92d98707d3335 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:18.074697 11039 tablet_bootstrap.cc:654] T a6e4030e824047ca9cc92d98707d3335 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
W20260924 00:21:18.076108 11459 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.4.65:49739 (request call id 65) took 1510 ms (client timeout 1000 ms). Trace:
W20260924 00:21:18.077505 10876 consensus_peers.cc:487] T 6bcfd729e9bd4040b3384ab24164e80d P c8626e300d0644b0908797ad61bd7cef -> Peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Couldn't send request to peer b10a9082a37b46f09d581944914c1670. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:33461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 00:21:18.079912 10876 consensus_peers.cc:487] T 5b43ce1fed0749b8b076b9b026914e07 P c8626e300d0644b0908797ad61bd7cef -> Peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Couldn't send request to peer b10a9082a37b46f09d581944914c1670. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:33461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 00:21:18.087599 11459 rpcz_store.cc:269] 0924 00:21:16.565710 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:16.565848 (+ 138us) service_pool.cc:227] Handling call
0924 00:21:18.073378 (+1507530us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b5400010754. stack: 00007f8e71958033 00007f8e71957f30 00007f8e714d9534 00000000004db2b4 00000000004db249 00007f8e78864af0 00007f8e788e7f2a 00007f8e7a7b3d6c 00007f8e7a8adc3e 00007f8e783e4fb4 00007f8e783e4f42 00007f8e783e4e71 00007f8e783e4dec 00007f8e783e40a2 00007f8e7668e94c 00007f8e7668dd86
0924 00:21:18.075934 (+ 2556us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3462821632}
I20260924 00:21:18.088631 11039 tablet_bootstrap.cc:492] T a6e4030e824047ca9cc92d98707d3335 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:18.089133 11039 ts_tablet_manager.cc:1397] T a6e4030e824047ca9cc92d98707d3335 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.021s user 0.009s sys 0.000s
W20260924 00:21:18.081579 11456 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.4.67:40723 (request call id 67) took 1375 ms (client timeout 1000 ms). Trace:
W20260924 00:21:18.090049 11456 rpcz_store.cc:269] 0924 00:21:16.706489 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:16.706625 (+ 136us) service_pool.cc:227] Handling call
0924 00:21:18.079296 (+1372671us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b5400010754. stack: 00007f8e71958033 00007f8e71957f30 00007f8e714d9534 00000000004db2b4 00000000004db249 00007f8e78864af0 00007f8e788e7f2a 00007f8e7a7b3d6c 00007f8e7a8adc3e 00007f8e783e4fb4 00007f8e783e4f42 00007f8e783e4e71 00007f8e783e4dec 00007f8e783e40a2 00007f8e7668e94c 00007f8e7668dd86
0924 00:21:18.081405 (+ 2109us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3152591616}
I20260924 00:21:18.082182 11944 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1d68651913494ce8afee3c136b2da9cc)
I20260924 00:21:18.091408 11039 raft_consensus.cc:348] T a6e4030e824047ca9cc92d98707d3335 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:18.091368 11944 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } }
I20260924 00:21:18.092012 11039 raft_consensus.cc:374] T a6e4030e824047ca9cc92d98707d3335 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:18.092232 11039 raft_consensus.cc:729] T a6e4030e824047ca9cc92d98707d3335 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:18.092753 11039 consensus_queue.cc:260] T a6e4030e824047ca9cc92d98707d3335 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:18.093778 11944 leader_election.cc:288] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1d68651913494ce8afee3c136b2da9cc (127.10.4.126:34479), b5aa3debe1cd45e19a6bbd0070913ac3 (127.10.4.125:34487)
I20260924 00:21:18.094637 11039 ts_tablet_manager.cc:1428] T a6e4030e824047ca9cc92d98707d3335 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
W20260924 00:21:18.086848 11463 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.4.66:33619 (request call id 63) took 1502 ms (client timeout 1000 ms). Trace:
W20260924 00:21:18.095185 11463 rpcz_store.cc:269] 0924 00:21:16.584474 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:16.584622 (+ 148us) service_pool.cc:227] Handling call
0924 00:21:18.084916 (+1500294us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b5400010754. stack: 00007f8e71958033 00007f8e71957f30 00007f8e714d9534 00000000004db2b4 00000000004db249 00007f8e78864af0 00007f8e788e7f2a 00007f8e7a7b3d6c 00007f8e7a8adc3e 00007f8e783e4fb4 00007f8e783e4f42 00007f8e783e4e71 00007f8e783e4dec 00007f8e783e40a2 00007f8e7668e94c 00007f8e7668dd86
0924 00:21:18.086656 (+ 1740us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3438610816}
I20260924 00:21:18.078068 11915 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d68651913494ce8afee3c136b2da9cc. Latest consensus state: current_term: 3 leader_uuid: "1d68651913494ce8afee3c136b2da9cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } } }
I20260924 00:21:18.096009 10309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" candidate_term: 4 candidate_status { last_received { term: 3 index: 454 } } ignore_live_leader: false dest_uuid: "1d68651913494ce8afee3c136b2da9cc" is_pre_election: true
I20260924 00:21:18.096428 11915 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:21:18.097566 11039 tablet_bootstrap.cc:492] T 8d34535febc94548b48b6b73e1ba85e0 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
W20260924 00:21:18.098024 11410 connection.cc:441] server connection from 127.10.4.65:49739 torn down before Call kudu.master.MasterService.TSHeartbeat from 127.10.4.65:49739 (request call id 65) could send its response
I20260924 00:21:18.100201 10384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" candidate_term: 4 candidate_status { last_received { term: 3 index: 454 } } ignore_live_leader: false dest_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" is_pre_election: true
I20260924 00:21:18.103446 11412 leader_election.cc:302] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f8bd0b3184f84f5f8da10c9c21472f92; no voters: 1d68651913494ce8afee3c136b2da9cc, b5aa3debe1cd45e19a6bbd0070913ac3
I20260924 00:21:18.103343 11974 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "1d68651913494ce8afee3c136b2da9cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } } }
I20260924 00:21:18.104521 11915 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 00:21:18.104111 11974 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:21:18.122368 11939 raft_consensus.cc:482] T 79d8dc7403f448df8cb960311eefe110 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:18.122774 11939 raft_consensus.cc:504] T 79d8dc7403f448df8cb960311eefe110 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:18.125434 11808 raft_consensus.cc:482] T b1c2eb7623f847a3a5b6964627ab94df P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:18.125892 11808 raft_consensus.cc:504] T b1c2eb7623f847a3a5b6964627ab94df P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:18.127439 11939 leader_election.cc:288] T 79d8dc7403f448df8cb960311eefe110 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:18.127564 11808 leader_election.cc:288] T b1c2eb7623f847a3a5b6964627ab94df P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
I20260924 00:21:18.130221 11039 tablet_bootstrap.cc:654] T 8d34535febc94548b48b6b73e1ba85e0 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
W20260924 00:21:18.130820 10876 leader_election.cc:334] T 79d8dc7403f448df8cb960311eefe110 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Network error: Client connection negotiation failed: client connection to 127.10.4.65:33461: connect: Connection refused (error 111)
W20260924 00:21:18.134397 10876 leader_election.cc:334] T b1c2eb7623f847a3a5b6964627ab94df P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Network error: Client connection negotiation failed: client connection to 127.10.4.65:33461: connect: Connection refused (error 111)
W20260924 00:21:18.142122 10876 consensus_peers.cc:487] T a53f06276dbb4d9f97a154a8ceb0278d P c8626e300d0644b0908797ad61bd7cef -> Peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Couldn't send request to peer b10a9082a37b46f09d581944914c1670. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:33461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 00:21:18.149127 11508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59602 (request call id 3) took 1627 ms. Trace:
I20260924 00:21:18.154335 11039 tablet_bootstrap.cc:492] T 8d34535febc94548b48b6b73e1ba85e0 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:18.154728 11039 ts_tablet_manager.cc:1397] T 8d34535febc94548b48b6b73e1ba85e0 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.057s user 0.007s sys 0.000s
I20260924 00:21:18.156796 11039 raft_consensus.cc:348] T 8d34535febc94548b48b6b73e1ba85e0 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:18.157434 11039 raft_consensus.cc:374] T 8d34535febc94548b48b6b73e1ba85e0 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:18.157686 11039 raft_consensus.cc:729] T 8d34535febc94548b48b6b73e1ba85e0 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:18.158402 11039 consensus_queue.cc:260] T 8d34535febc94548b48b6b73e1ba85e0 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:18.160277 11039 ts_tablet_manager.cc:1428] T 8d34535febc94548b48b6b73e1ba85e0 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260924 00:21:18.161787 11939 raft_consensus.cc:482] T 8d34535febc94548b48b6b73e1ba85e0 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:18.162217 11939 raft_consensus.cc:504] T 8d34535febc94548b48b6b73e1ba85e0 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:18.163849 11939 leader_election.cc:288] T 8d34535febc94548b48b6b73e1ba85e0 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
W20260924 00:21:18.168119 10876 leader_election.cc:334] T 8d34535febc94548b48b6b73e1ba85e0 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Network error: Client connection negotiation failed: client connection to 127.10.4.65:33461: connect: Connection refused (error 111)
I20260924 00:21:18.184561 11888 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:21:18.185937 11939 raft_consensus.cc:482] T 8e628abffd9545b391c25f83c9903732 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:18.186323 11939 raft_consensus.cc:504] T 8e628abffd9545b391c25f83c9903732 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:18.188045 11939 leader_election.cc:288] T 8e628abffd9545b391c25f83c9903732 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
W20260924 00:21:18.192081 10876 leader_election.cc:334] T 8e628abffd9545b391c25f83c9903732 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Network error: Client connection negotiation failed: client connection to 127.10.4.65:33461: connect: Connection refused (error 111)
I20260924 00:21:18.206631 11039 tablet_bootstrap.cc:492] T bc2bbf10bfbc46639f2bf288657ff631 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:18.224985 11039 tablet_bootstrap.cc:654] T bc2bbf10bfbc46639f2bf288657ff631 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
W20260924 00:21:18.234452 10876 consensus_peers.cc:487] T d32a0c74b527451fb723e3851d5d055b P c8626e300d0644b0908797ad61bd7cef -> Peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Couldn't send request to peer b10a9082a37b46f09d581944914c1670. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:33461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 00:21:18.277940 11039 tablet_bootstrap.cc:492] T bc2bbf10bfbc46639f2bf288657ff631 P c8626e300d0644b0908797ad61bd7cef: No bootstrap required, opened a new log
I20260924 00:21:18.278411 11039 ts_tablet_manager.cc:1397] T bc2bbf10bfbc46639f2bf288657ff631 P c8626e300d0644b0908797ad61bd7cef: Time spent bootstrapping tablet: real 0.072s user 0.005s sys 0.004s
I20260924 00:21:18.280449 11039 raft_consensus.cc:348] T bc2bbf10bfbc46639f2bf288657ff631 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:18.281328 11039 raft_consensus.cc:374] T bc2bbf10bfbc46639f2bf288657ff631 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:18.281692 11039 raft_consensus.cc:729] T bc2bbf10bfbc46639f2bf288657ff631 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8626e300d0644b0908797ad61bd7cef, State: Initialized, Role: FOLLOWER
I20260924 00:21:18.282371 11039 consensus_queue.cc:260] T bc2bbf10bfbc46639f2bf288657ff631 P c8626e300d0644b0908797ad61bd7cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:18.284621 11039 ts_tablet_manager.cc:1428] T bc2bbf10bfbc46639f2bf288657ff631 P c8626e300d0644b0908797ad61bd7cef: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260924 00:21:18.285492 11039 tablet_bootstrap.cc:492] T cf0f0f016d074b1fb78d9b427c67fe86 P c8626e300d0644b0908797ad61bd7cef: Bootstrap starting.
I20260924 00:21:18.299019 11799 raft_consensus.cc:482] T c03aa0298c0c4fe580c73cfc61f971a3 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:18.299456 11799 raft_consensus.cc:504] T c03aa0298c0c4fe580c73cfc61f971a3 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:18.301311 11799 leader_election.cc:288] T c03aa0298c0c4fe580c73cfc61f971a3 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
W20260924 00:21:18.305768 10876 leader_election.cc:334] T c03aa0298c0c4fe580c73cfc61f971a3 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Network error: Client connection negotiation failed: client connection to 127.10.4.65:33461: connect: Connection refused (error 111)
W20260924 00:21:18.319533 10878 connection.cc:535] client connection to 127.10.4.66:35359 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260924 00:21:18.319692 11406 connection.cc:535] server connection from 127.10.4.66:33619 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260924 00:21:18.323530 11799 raft_consensus.cc:482] T a6e4030e824047ca9cc92d98707d3335 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:18.323944 11799 raft_consensus.cc:504] T a6e4030e824047ca9cc92d98707d3335 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48fbd603abe146c585d15196e1b5be08" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 35359 } } peers { permanent_uuid: "c8626e300d0644b0908797ad61bd7cef" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 41301 } } peers { permanent_uuid: "b10a9082a37b46f09d581944914c1670" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 33461 } }
I20260924 00:21:18.325717 11799 leader_election.cc:288] T a6e4030e824047ca9cc92d98707d3335 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359), b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461)
W20260924 00:21:18.329949 10876 leader_election.cc:334] T a6e4030e824047ca9cc92d98707d3335 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Network error: Client connection negotiation failed: client connection to 127.10.4.65:33461: connect: Connection refused (error 111)
I20260924 00:21:18.330565 11039 tablet_bootstrap.cc:654] T cf0f0f016d074b1fb78d9b427c67fe86 P c8626e300d0644b0908797ad61bd7cef: Neither blocks nor log segments found. Creating new log.
W20260924 00:21:18.336330 11972 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:21:18.330887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.4.66:35359 (local address 127.10.4.67:38515)
0924 00:21:18.332603 (+ 1716us) negotiation.cc:107] Waiting for socket to connect
0924 00:21:18.332668 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0924 00:21:18.333122 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:21:18.335613 (+ 2491us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.4.66:35359: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1327}
W20260924 00:21:18.337343 10878 consensus_peers.cc:487] T 5b0345252c3c496db2987c30adce3da7 P c8626e300d0644b0908797ad61bd7cef -> Peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Couldn't send request to peer 48fbd603abe146c585d15196e1b5be08. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:35359: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
W20260924 00:21:18.338310 10878 leader_election.cc:334] T 915e8072851c4d2b90f8af12b1dc58e0 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Network error: Client connection negotiation failed: client connection to 127.10.4.66:35359: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260924 00:21:18.338694 10878 leader_election.cc:302] T 915e8072851c4d2b90f8af12b1dc58e0 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670
W20260924 00:21:18.339584 10878 leader_election.cc:334] T 79d8dc7403f448df8cb960311eefe110 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Network error: Client connection negotiation failed: client connection to 127.10.4.66:35359: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260924 00:21:18.340395 10878 leader_election.cc:302] T 79d8dc7403f448df8cb960311eefe110 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670
I20260924 00:21:18.340032 11799 raft_consensus.cc:2726] T 915e8072851c4d2b90f8af12b1dc58e0 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260924 00:21:18.342010 10878 leader_election.cc:334] T b1c2eb7623f847a3a5b6964627ab94df P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Network error: Client connection negotiation failed: client connection to 127.10.4.66:35359: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260924 00:21:18.342434 11799 raft_consensus.cc:2726] T 79d8dc7403f448df8cb960311eefe110 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:18.342962 10878 leader_election.cc:302] T b1c2eb7623f847a3a5b6964627ab94df P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670
I20260924 00:21:18.343588 11799 raft_consensus.cc:2726] T b1c2eb7623f847a3a5b6964627ab94df P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260924 00:21:18.344270 10878 leader_election.cc:334] T 8d34535febc94548b48b6b73e1ba85e0 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Network error: Client connection negotiation failed: client connection to 127.10.4.66:35359: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260924 00:21:18.344642 10878 leader_election.cc:302] T 8d34535febc94548b48b6b73e1ba85e0 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670
I20260924 00:21:18.345332 11799 raft_consensus.cc:2726] T 8d34535febc94548b48b6b73e1ba85e0 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260924 00:21:18.346961 10878 leader_election.cc:334] T 8e628abffd9545b391c25f83c9903732 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Network error: Client connection negotiation failed: client connection to 127.10.4.66:35359: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260924 00:21:18.347352 10878 leader_election.cc:302] T 8e628abffd9545b391c25f83c9903732 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670
I20260924 00:21:18.347946 11799 raft_consensus.cc:2726] T 8e628abffd9545b391c25f83c9903732 P c8626e300d0644b0908797ad61bd7cef [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260924 00:21:18.348701 10878 consensus_peers.cc:487] T a53f06276dbb4d9f97a154a8ceb0278d P c8626e300d0644b0908797ad61bd7cef -> Peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Couldn't send request to peer 48fbd603abe146c585d15196e1b5be08. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:35359: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
W20260924 00:21:18.358393 10361 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet f220c1fadaf046f6bf085fcdf87109dd (table default.Table_06b78e500bd24799a8632eac4b4b108c [id=ed8b9a9a78d7481993cd6acca559d520], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet f220c1fadaf046f6bf085fcdf87109dd has no consensus state
W20260924 00:21:18.360184 10876 consensus_peers.cc:487] T f85ac5b0fd2348d88e1076b781424ab3 P c8626e300d0644b0908797ad61bd7cef -> Peer b10a9082a37b46f09d581944914c1670 (127.10.4.65:33461): Couldn't send request to peer b10a9082a37b46f09d581944914c1670. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:33461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 00:21:18.368979 10878 consensus_peers.cc:487] T 5b43ce1fed0749b8b076b9b026914e07 P c8626e300d0644b0908797ad61bd7cef -> Peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Couldn't send request to peer 48fbd603abe146c585d15196e1b5be08. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:35359: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
W20260924 00:21:18.369357 10878 leader_election.cc:334] T c03aa0298c0c4fe580c73cfc61f971a3 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 48fbd603abe146c585d15196e1b5be08 (127.10.4.66:35359): Network error: Client connection negotiation failed: client connection to 127.10.4.66:35359: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260924 00:21:18.369699 10878 leader_election.cc:302] T c03aa0298c0c4fe580c73cfc61f971a3 P c8626e300d0644b0908797ad61bd7cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8626e300d0644b0908797ad61bd7cef; no voters: 48fbd603abe146c585d15196e1b5be08, b10a9082a37b46f09d581944914c1670
I20260924 00:21:18.371271 10257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu with pid 10855
I20260924 00:21:18.377985 11888 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1d68651913494ce8afee3c136b2da9cc: Loaded TSK: 0
I20260924 00:21:18.379423 11888 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:21:18.461113 10257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu with pid 10266
I20260924 00:21:18.547556 10257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu with pid 10336
I20260924 00:21:12.174240 11510 rpcz_store.cc:276] 0924 00:21:08.445073 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:08.445244 (+ 171us) service_pool.cc:227] Handling call
0924 00:21:08.890429 (+445185us) raft_consensus.cc:1457] Updating replica for 317 ops
0924 00:21:09.013141 (+122712us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 00:21:09.013196 (+ 55us) raft_consensus.cc:1525] Triggering prepare for 317 ops
0924 00:21:09.017982 (+ 4786us) write_op.cc:270] Start()
0924 00:21:09.018385 (+ 403us) write_op.cc:276] Timestamp: P: 1790209255130762 usec, L: 0
0924 00:21:09.103211 (+ 84826us) write_op.cc:270] Start()
0924 00:21:09.103322 (+ 111us) write_op.cc:276] Timestamp: P: 1790209255137047 usec, L: 0
0924 00:21:09.103867 (+ 545us) write_op.cc:270] Start()
0924 00:21:09.103997 (+ 130us) write_op.cc:276] Timestamp: P: 1790209255138852 usec, L: 0
0924 00:21:09.111980 (+ 7983us) write_op.cc:270] Start()
0924 00:21:09.112086 (+ 106us) write_op.cc:276] Timestamp: P: 1790209255141696 usec, L: 0
0924 00:21:09.112615 (+ 529us) write_op.cc:270] Start()
0924 00:21:09.112692 (+ 77us) write_op.cc:276] Timestamp: P: 1790209255183655 usec, L: 0
0924 00:21:09.114923 (+ 2231us) write_op.cc:270] Start()
0924 00:21:09.115014 (+ 91us) write_op.cc:276] Timestamp: P: 1790209255185783 usec, L: 0
0924 00:21:09.115443 (+ 429us) write_op.cc:270] Start()
0924 00:21:09.115576 (+ 133us) write_op.cc:276] Timestamp: P: 1790209255189953 usec, L: 0
0924 00:21:09.115958 (+ 382us) write_op.cc:270] Start()
0924 00:21:09.116026 (+ 68us) write_op.cc:276] Timestamp: P: 1790209255192808 usec, L: 0
0924 00:21:09.116437 (+ 411us) write_op.cc:270] Start()
0924 00:21:09.116512 (+ 75us) write_op.cc:276] Timestamp: P: 1790209255195858 usec, L: 0
0924 00:21:09.116893 (+ 381us) write_op.cc:270] Start()
0924 00:21:09.136731 (+ 19838us) write_op.cc:276] Timestamp: P: 1790209255235525 usec, L: 0
0924 00:21:09.137338 (+ 607us) write_op.cc:270] Start()
0924 00:21:09.137438 (+ 100us) write_op.cc:276] Timestamp: P: 1790209255349902 usec, L: 0
0924 00:21:09.137897 (+ 459us) write_op.cc:270] Start()
0924 00:21:09.137995 (+ 98us) write_op.cc:276] Timestamp: P: 1790209255363496 usec, L: 0
0924 00:21:09.138981 (+ 986us) write_op.cc:270] Start()
0924 00:21:09.139057 (+ 76us) write_op.cc:276] Timestamp: P: 1790209255378272 usec, L: 0
0924 00:21:09.139483 (+ 426us) write_op.cc:270] Start()
0924 00:21:09.139557 (+ 74us) write_op.cc:276] Timestamp: P: 1790209255420691 usec, L: 0
0924 00:21:09.320786 (+181229us) write_op.cc:270] Start()
0924 00:21:09.320871 (+ 85us) write_op.cc:276] Timestamp: P: 1790209255427802 usec, L: 0
0924 00:21:09.321364 (+ 493us) write_op.cc:270] Start()
0924 00:21:09.321443 (+ 79us) write_op.cc:276] Timestamp: P: 1790209255442449 usec, L: 0
0924 00:21:09.322501 (+ 1058us) write_op.cc:270] Start()
0924 00:21:09.322580 (+ 79us) write_op.cc:276] Timestamp: P: 1790209255456596 usec, L: 0
0924 00:21:09.322996 (+ 416us) write_op.cc:270] Start()
0924 00:21:09.323084 (+ 88us) write_op.cc:276] Timestamp: P: 1790209255461767 usec, L: 0
0924 00:21:09.323480 (+ 396us) write_op.cc:270] Start()
0924 00:21:09.323562 (+ 82us) write_op.cc:276] Timestamp: P: 1790209255481691 usec, L: 0
0924 00:21:09.323953 (+ 391us) write_op.cc:270] Start()
0924 00:21:09.324028 (+ 75us) write_op.cc:276] Timestamp: P: 1790209255524902 usec, L: 0
0924 00:21:09.324399 (+ 371us) write_op.cc:270] Start()
0924 00:21:09.324473 (+ 74us) write_op.cc:276] Timestamp: P: 1790209255536748 usec, L: 0
0924 00:21:09.324877 (+ 404us) write_op.cc:270] Start()
0924 00:21:09.553616 (+228739us) write_op.cc:276] Timestamp: P: 1790209255563095 usec, L: 0
0924 00:21:09.674217 (+120601us) write_op.cc:270] Start()
0924 00:21:09.674312 (+ 95us) write_op.cc:276] Timestamp: P: 1790209255629941 usec, L: 0
0924 00:21:09.674734 (+ 422us) write_op.cc:270] Start()
0924 00:21:09.674846 (+ 112us) write_op.cc:276] Timestamp: P: 1790209255632062 usec, L: 0
0924 00:21:09.675456 (+ 610us) write_op.cc:270] Start()
0924 00:21:09.675533 (+ 77us) write_op.cc:276] Timestamp: P: 1790209255644702 usec, L: 0
0924 00:21:09.700888 (+ 25355us) write_op.cc:270] Start()
0924 00:21:09.701007 (+ 119us) write_op.cc:276] Timestamp: P: 1790209255656164 usec, L: 0
0924 00:21:09.701502 (+ 495us) write_op.cc:270] Start()
0924 00:21:09.701580 (+ 78us) write_op.cc:276] Timestamp: P: 1790209255682478 usec, L: 0
0924 00:21:09.709530 (+ 7950us) write_op.cc:270] Start()
0924 00:21:09.709611 (+ 81us) write_op.cc:276] Timestamp: P: 1790209255696581 usec, L: 0
0924 00:21:09.710047 (+ 436us) write_op.cc:270] Start()
0924 00:21:09.710126 (+ 79us) write_op.cc:276] Timestamp: P: 1790209255703630 usec, L: 0
0924 00:21:09.710785 (+ 659us) write_op.cc:270] Start()
0924 00:21:09.710862 (+ 77us) write_op.cc:276] Timestamp: P: 1790209255732786 usec, L: 0
0924 00:21:09.711282 (+ 420us) write_op.cc:270] Start()
0924 00:21:09.711359 (+ 77us) write_op.cc:276] Timestamp: P: 1790209255736021 usec, L: 0
0924 00:21:09.711802 (+ 443us) write_op.cc:270] Start()
0924 00:21:09.711883 (+ 81us) write_op.cc:276] Timestamp: P: 1790209255758747 usec, L: 0
0924 00:21:09.712373 (+ 490us) write_op.cc:270] Start()
0924 00:21:09.712446 (+ 73us) write_op.cc:276] Timestamp: P: 1790209255794353 usec, L: 0
0924 00:21:09.712837 (+ 391us) write_op.cc:270] Start()
0924 00:21:09.712909 (+ 72us) write_op.cc:276] Timestamp: P: 1790209255854204 usec, L: 0
0924 00:21:09.751847 (+ 38938us) write_op.cc:270] Start()
0924 00:21:09.751945 (+ 98us) write_op.cc:276] Timestamp: P: 1790209255876391 usec, L: 0
0924 00:21:09.752397 (+ 452us) write_op.cc:270] Start()
0924 00:21:09.752475 (+ 78us) write_op.cc:276] Timestamp: P: 1790209255914401 usec, L: 0
0924 00:21:09.752911 (+ 436us) write_op.cc:270] Start()
0924 00:21:09.769064 (+ 16153us) write_op.cc:276] Timestamp: P: 1790209255919623 usec, L: 0
0924 00:21:09.769614 (+ 550us) write_op.cc:270] Start()
0924 00:21:09.769698 (+ 84us) write_op.cc:276] Timestamp: P: 1790209255933483 usec, L: 0
0924 00:21:09.770153 (+ 455us) write_op.cc:270] Start()
0924 00:21:09.770254 (+ 101us) write_op.cc:276] Timestamp: P: 1790209255971963 usec, L: 0
0924 00:21:09.770684 (+ 430us) write_op.cc:270] Start()
0924 00:21:09.770767 (+ 83us) write_op.cc:276] Timestamp: P: 1790209255974009 usec, L: 0
0924 00:21:09.771175 (+ 408us) write_op.cc:270] Start()
0924 00:21:09.771246 (+ 71us) write_op.cc:276] Timestamp: P: 1790209256045910 usec, L: 0
0924 00:21:09.771639 (+ 393us) write_op.cc:270] Start()
0924 00:21:09.771710 (+ 71us) write_op.cc:276] Timestamp: P: 1790209256070521 usec, L: 0
0924 00:21:09.772118 (+ 408us) write_op.cc:270] Start()
0924 00:21:09.772198 (+ 80us) write_op.cc:276] Timestamp: P: 1790209256078603 usec, L: 0
0924 00:21:09.772599 (+ 401us) write_op.cc:270] Start()
0924 00:21:09.772681 (+ 82us) write_op.cc:276] Timestamp: P: 1790209256082421 usec, L: 0
0924 00:21:09.810622 (+ 37941us) write_op.cc:270] Start()
0924 00:21:09.810823 (+ 201us) write_op.cc:276] Timestamp: P: 1790209256113530 usec, L: 0
0924 00:21:09.811459 (+ 636us) write_op.cc:270] Start()
0924 00:21:09.811546 (+ 87us) write_op.cc:276] Timestamp: P: 1790209256153663 usec, L: 0
0924 00:21:09.812008 (+ 462us) write_op.cc:270] Start()
0924 00:21:09.812086 (+ 78us) write_op.cc:276] Timestamp: P: 1790209256219407 usec, L: 0
0924 00:21:09.812520 (+ 434us) write_op.cc:270] Start()
0924 00:21:09.812669 (+ 149us) write_op.cc:276] Timestamp: P: 1790209256252227 usec, L: 0
0924 00:21:09.813182 (+ 513us) write_op.cc:270] Start()
0924 00:21:09.813259 (+ 77us) write_op.cc:276] Timestamp: P: 1790209256262323 usec, L: 0
0924 00:21:09.813689 (+ 430us) write_op.cc:270] Start()
0924 00:21:09.813778 (+ 89us) write_op.cc:276] Timestamp: P: 1790209256276650 usec, L: 0
0924 00:21:09.814173 (+ 395us) write_op.cc:270] Start()
0924 00:21:09.814242 (+ 69us) write_op.cc:276] Timestamp: P: 1790209256306725 usec, L: 0
0924 00:21:09.814641 (+ 399us) write_op.cc:270] Start()
0924 00:21:09.814736 (+ 95us) write_op.cc:276] Timestamp: P: 1790209256389299 usec, L: 0
0924 00:21:09.815134 (+ 398us) write_op.cc:270] Start()
0924 00:21:09.815206 (+ 72us) write_op.cc:276] Timestamp: P: 1790209256450485 usec, L: 0
0924 00:21:09.815597 (+ 391us) write_op.cc:270] Start()
0924 00:21:09.815673 (+ 76us) write_op.cc:276] Timestamp: P: 1790209256501221 usec, L: 0
0924 00:21:09.816077 (+ 404us) write_op.cc:270] Start()
0924 00:21:09.816148 (+ 71us) write_op.cc:276] Timestamp: P: 1790209256503112 usec, L: 0
0924 00:21:09.993898 (+177750us) write_op.cc:270] Start()
0924 00:21:09.993994 (+ 96us) write_op.cc:276] Timestamp: P: 1790209256565085 usec, L: 0
0924 00:21:09.994510 (+ 516us) write_op.cc:270] Start()
0924 00:21:09.994589 (+ 79us) write_op.cc:276] Timestamp: P: 1790209256625897 usec, L: 0
0924 00:21:09.995033 (+ 444us) write_op.cc:270] Start()
0924 00:21:09.995173 (+ 140us) write_op.cc:276] Timestamp: P: 1790209256630920 usec, L: 0
0924 00:21:10.008487 (+ 13314us) write_op.cc:270] Start()
0924 00:21:10.008582 (+ 95us) write_op.cc:276] Timestamp: P: 1790209256642150 usec, L: 0
0924 00:21:10.009122 (+ 540us) write_op.cc:270] Start()
0924 00:21:10.009277 (+ 155us) write_op.cc:276] Timestamp: P: 1790209256648370 usec, L: 0
0924 00:21:10.009756 (+ 479us) write_op.cc:270] Start()
0924 00:21:10.009839 (+ 83us) write_op.cc:276] Timestamp: P: 1790209256714426 usec, L: 0
0924 00:21:10.017149 (+ 7310us) write_op.cc:270] Start()
0924 00:21:10.017236 (+ 87us) write_op.cc:276] Timestamp: P: 1790209256774133 usec, L: 0
0924 00:21:10.017772 (+ 536us) write_op.cc:270] Start()
0924 00:21:10.017934 (+ 162us) write_op.cc:276] Timestamp: P: 1790209256775984 usec, L: 0
0924 00:21:10.018539 (+ 605us) write_op.cc:270] Start()
0924 00:21:10.018618 (+ 79us) write_op.cc:276] Timestamp: P: 1790209256793830 usec, L: 0
0924 00:21:10.019211 (+ 593us) write_op.cc:270] Start()
0924 00:21:10.019288 (+ 77us) write_op.cc:276] Timestamp: P: 1790209256801507 usec, L: 0
0924 00:21:10.019795 (+ 507us) write_op.cc:270] Start()
0924 00:21:10.019869 (+ 74us) write_op.cc:276] Timestamp: P: 1790209256803112 usec, L: 0
0924 00:21:10.020261 (+ 392us) write_op.cc:270] Start()
0924 00:21:10.020330 (+ 69us) write_op.cc:276] Timestamp: P: 1790209256843478 usec, L: 0
0924 00:21:10.020731 (+ 401us) write_op.cc:270] Start()
0924 00:21:10.020813 (+ 82us) write_op.cc:276] Timestamp: P: 1790209256874905 usec, L: 0
0924 00:21:10.021484 (+ 671us) write_op.cc:270] Start()
0924 00:21:10.021554 (+ 70us) write_op.cc:276] Timestamp: P: 1790209256902691 usec, L: 0
0924 00:21:10.021937 (+ 383us) write_op.cc:270] Start()
0924 00:21:10.022004 (+ 67us) write_op.cc:276] Timestamp: P: 1790209256904710 usec, L: 0
0924 00:21:10.022354 (+ 350us) write_op.cc:270] Start()
0924 00:21:10.022420 (+ 66us) write_op.cc:276] Timestamp: P: 1790209256970368 usec, L: 0
0924 00:21:10.022781 (+ 361us) write_op.cc:270] Start()
0924 00:21:10.022848 (+ 67us) write_op.cc:276] Timestamp: P: 1790209256972371 usec, L: 0
0924 00:21:10.023197 (+ 349us) write_op.cc:270] Start()
0924 00:21:10.023272 (+ 75us) write_op.cc:276] Timestamp: P: 1790209257086989 usec, L: 0
0924 00:21:10.023627 (+ 355us) write_op.cc:270] Start()
0924 00:21:10.023700 (+ 73us) write_op.cc:276] Timestamp: P: 1790209257091008 usec, L: 0
0924 00:21:10.024098 (+ 398us) write_op.cc:270] Start()
0924 00:21:10.024171 (+ 73us) write_op.cc:276] Timestamp: P: 1790209257150732 usec, L: 0
0924 00:21:10.024554 (+ 383us) write_op.cc:270] Start()
0924 00:21:10.024625 (+ 71us) write_op.cc:276] Timestamp: P: 1790209257157810 usec, L: 0
0924 00:21:10.025072 (+ 447us) write_op.cc:270] Start()
0924 00:21:10.025150 (+ 78us) write_op.cc:276] Timestamp: P: 1790209257253314 usec, L: 0
0924 00:21:10.033653 (+ 8503us) write_op.cc:270] Start()
0924 00:21:10.033745 (+ 92us) write_op.cc:276] Timestamp: P: 1790209257274130 usec, L: 0
0924 00:21:10.034150 (+ 405us) write_op.cc:270] Start()
0924 00:21:10.034294 (+ 144us) write_op.cc:276] Timestamp: P: 1790209257281995 usec, L: 0
0924 00:21:10.034717 (+ 423us) write_op.cc:270] Start()
0924 00:21:10.034787 (+ 70us) write_op.cc:276] Timestamp: P: 1790209257356268 usec, L: 0
0924 00:21:10.035156 (+ 369us) write_op.cc:270] Start()
0924 00:21:10.035231 (+ 75us) write_op.cc:276] Timestamp: P: 1790209257455438 usec, L: 0
0924 00:21:10.035622 (+ 391us) write_op.cc:270] Start()
0924 00:21:10.036779 (+ 1157us) write_op.cc:276] Timestamp: P: 1790209257464128 usec, L: 0
0924 00:21:10.037343 (+ 564us) write_op.cc:270] Start()
0924 00:21:10.037417 (+ 74us) write_op.cc:276] Timestamp: P: 1790209257519351 usec, L: 0
0924 00:21:10.037822 (+ 405us) write_op.cc:270] Start()
0924 00:21:10.037895 (+ 73us) write_op.cc:276] Timestamp: P: 1790209257536269 usec, L: 0
0924 00:21:10.038298 (+ 403us) write_op.cc:270] Start()
0924 00:21:10.038369 (+ 71us) write_op.cc:276] Timestamp: P: 1790209257542111 usec, L: 0
0924 00:21:10.038795 (+ 426us) write_op.cc:270] Start()
0924 00:21:10.038868 (+ 73us) write_op.cc:276] Timestamp: P: 1790209257580834 usec, L: 0
0924 00:21:10.041015 (+ 2147us) write_op.cc:270] Start()
0924 00:21:10.041101 (+ 86us) write_op.cc:276] Timestamp: P: 1790209257596006 usec, L: 0
0924 00:21:10.041789 (+ 688us) write_op.cc:270] Start()
0924 00:21:10.041866 (+ 77us) write_op.cc:276] Timestamp: P: 1790209257613908 usec, L: 0
0924 00:21:10.043403 (+ 1537us) write_op.cc:270] Start()
0924 00:21:10.043489 (+ 86us) write_op.cc:276] Timestamp: P: 1790209257679916 usec, L: 0
0924 00:21:10.048329 (+ 4840us) write_op.cc:270] Start()
0924 00:21:10.048429 (+ 100us) write_op.cc:276] Timestamp: P: 1790209257708137 usec, L: 0
0924 00:21:10.048889 (+ 460us) write_op.cc:270] Start()
0924 00:21:10.049006 (+ 117us) write_op.cc:276] Timestamp: P: 1790209257716841 usec, L: 0
0924 00:21:10.049450 (+ 444us) write_op.cc:270] Start()
0924 00:21:10.049523 (+ 73us) write_op.cc:276] Timestamp: P: 1790209257757614 usec, L: 0
0924 00:21:10.049930 (+ 407us) write_op.cc:270] Start()
0924 00:21:10.050001 (+ 71us) write_op.cc:276] Timestamp: P: 1790209257759808 usec, L: 0
0924 00:21:10.050416 (+ 415us) write_op.cc:270] Start()
0924 00:21:10.050490 (+ 74us) write_op.cc:276] Timestamp: P: 1790209257991419 usec, L: 0
0924 00:21:10.050900 (+ 410us) write_op.cc:270] Start()
0924 00:21:10.050985 (+ 85us) write_op.cc:276] Timestamp: P: 1790209258016853 usec, L: 0
0924 00:21:10.051486 (+ 501us) write_op.cc:270] Start()
0924 00:21:10.051568 (+ 82us) write_op.cc:276] Timestamp: P: 1790209258205877 usec, L: 0
0924 00:21:10.051991 (+ 423us) write_op.cc:270] Start()
0924 00:21:10.052064 (+ 73us) write_op.cc:276] Timestamp: P: 1790209258208517 usec, L: 0
0924 00:21:10.083982 (+ 31918us) write_op.cc:270] Start()
0924 00:21:10.084198 (+ 216us) write_op.cc:276] Timestamp: P: 1790209258311599 usec, L: 0
0924 00:21:10.085007 (+ 809us) write_op.cc:270] Start()
0924 00:21:10.085088 (+ 81us) write_op.cc:276] Timestamp: P: 1790209258316368 usec, L: 0
0924 00:21:10.085648 (+ 560us) write_op.cc:270] Start()
0924 00:21:10.085734 (+ 86us) write_op.cc:276] Timestamp: P: 1790209258387535 usec, L: 0
0924 00:21:10.086175 (+ 441us) write_op.cc:270] Start()
0924 00:21:10.086255 (+ 80us) write_op.cc:276] Timestamp: P: 1790209258421178 usec, L: 0
0924 00:21:10.086734 (+ 479us) write_op.cc:270] Start()
0924 00:21:10.086817 (+ 83us) write_op.cc:276] Timestamp: P: 1790209258622709 usec, L: 0
0924 00:21:10.087235 (+ 418us) write_op.cc:270] Start()
0924 00:21:10.087317 (+ 82us) write_op.cc:276] Timestamp: P: 1790209258624557 usec, L: 0
0924 00:21:10.087725 (+ 408us) write_op.cc:270] Start()
0924 00:21:10.087801 (+ 76us) write_op.cc:276] Timestamp: P: 1790209258657485 usec, L: 0
0924 00:21:10.088212 (+ 411us) write_op.cc:270] Start()
0924 00:21:10.088285 (+ 73us) write_op.cc:276] Timestamp: P: 1790209258659195 usec, L: 0
0924 00:21:10.088687 (+ 402us) write_op.cc:270] Start()
0924 00:21:10.088768 (+ 81us) write_op.cc:276] Timestamp: P: 1790209258806125 usec, L: 0
0924 00:21:10.107499 (+ 18731us) write_op.cc:270] Start()
0924 00:21:10.107598 (+ 99us) write_op.cc:276] Timestamp: P: 1790209258852312 usec, L: 0
0924 00:21:10.108084 (+ 486us) write_op.cc:270] Start()
0924 00:21:10.108161 (+ 77us) write_op.cc:276] Timestamp: P: 1790209258906523 usec, L: 0
0924 00:21:10.108571 (+ 410us) write_op.cc:270] Start()
0924 00:21:10.108650 (+ 79us) write_op.cc:276] Timestamp: P: 1790209259021458 usec, L: 0
0924 00:21:10.109077 (+ 427us) write_op.cc:270] Start()
0924 00:21:10.109165 (+ 88us) write_op.cc:276] Timestamp: P: 1790209259031802 usec, L: 0
0924 00:21:10.109586 (+ 421us) write_op.cc:270] Start()
0924 00:21:10.109656 (+ 70us) write_op.cc:276] Timestamp: P: 1790209259071759 usec, L: 0
0924 00:21:10.110072 (+ 416us) write_op.cc:270] Start()
0924 00:21:10.110142 (+ 70us) write_op.cc:276] Timestamp: P: 1790209259119394 usec, L: 0
0924 00:21:10.112292 (+ 2150us) write_op.cc:270] Start()
0924 00:21:10.112368 (+ 76us) write_op.cc:276] Timestamp: P: 1790209259146392 usec, L: 0
0924 00:21:10.112794 (+ 426us) write_op.cc:270] Start()
0924 00:21:10.112871 (+ 77us) write_op.cc:276] Timestamp: P: 1790209259161967 usec, L: 0
0924 00:21:10.113296 (+ 425us) write_op.cc:270] Start()
0924 00:21:10.113373 (+ 77us) write_op.cc:276] Timestamp: P: 1790209259169903 usec, L: 0
0924 00:21:10.113867 (+ 494us) write_op.cc:270] Start()
0924 00:21:10.113939 (+ 72us) write_op.cc:276] Timestamp: P: 1790209259174716 usec, L: 0
0924 00:21:10.114336 (+ 397us) write_op.cc:270] Start()
0924 00:21:10.114409 (+ 73us) write_op.cc:276] Timestamp: P: 1790209259278084 usec, L: 0
0924 00:21:10.155440 (+ 41031us) write_op.cc:270] Start()
0924 00:21:10.155536 (+ 96us) write_op.cc:276] Timestamp: P: 1790209259290113 usec, L: 0
0924 00:21:10.155983 (+ 447us) write_op.cc:270] Start()
0924 00:21:10.156056 (+ 73us) write_op.cc:276] Timestamp: P: 1790209259306821 usec, L: 0
0924 00:21:10.156444 (+ 388us) write_op.cc:270] Start()
0924 00:21:10.156513 (+ 69us) write_op.cc:276] Timestamp: P: 1790209259357564 usec, L: 0
0924 00:21:10.156937 (+ 424us) write_op.cc:270] Start()
0924 00:21:10.163661 (+ 6724us) write_op.cc:276] Timestamp: P: 1790209259359250 usec, L: 0
0924 00:21:10.164270 (+ 609us) write_op.cc:270] Start()
0924 00:21:10.164349 (+ 79us) write_op.cc:276] Timestamp: P: 1790209259438088 usec, L: 0
0924 00:21:10.164836 (+ 487us) write_op.cc:270] Start()
0924 00:21:10.164916 (+ 80us) write_op.cc:276] Timestamp: P: 1790209259545800 usec, L: 0
0924 00:21:10.165368 (+ 452us) write_op.cc:270] Start()
0924 00:21:10.165442 (+ 74us) write_op.cc:276] Timestamp: P: 1790209259575200 usec, L: 0
0924 00:21:10.165871 (+ 429us) write_op.cc:270] Start()
0924 00:21:10.165945 (+ 74us) write_op.cc:276] Timestamp: P: 1790209259597856 usec, L: 0
0924 00:21:10.166351 (+ 406us) write_op.cc:270] Start()
0924 00:21:10.166427 (+ 76us) write_op.cc:276] Timestamp: P: 1790209259620333 usec, L: 0
0924 00:21:10.170828 (+ 4401us) write_op.cc:270] Start()
0924 00:21:10.170915 (+ 87us) write_op.cc:276] Timestamp: P: 1790209259629138 usec, L: 0
0924 00:21:10.171346 (+ 431us) write_op.cc:270] Start()
0924 00:21:10.171424 (+ 78us) write_op.cc:276] Timestamp: P: 1790209259637925 usec, L: 0
0924 00:21:10.172080 (+ 656us) write_op.cc:270] Start()
0924 00:21:10.172155 (+ 75us) write_op.cc:276] Timestamp: P: 1790209259710434 usec, L: 0
0924 00:21:10.172553 (+ 398us) write_op.cc:270] Start()
0924 00:21:10.172626 (+ 73us) write_op.cc:276] Timestamp: P: 1790209259724118 usec, L: 0
0924 00:21:10.173037 (+ 411us) write_op.cc:270] Start()
0924 00:21:10.173185 (+ 148us) write_op.cc:276] Timestamp: P: 1790209259838665 usec, L: 0
0924 00:21:10.173545 (+ 360us) write_op.cc:270] Start()
0924 00:21:10.173603 (+ 58us) write_op.cc:276] Timestamp: P: 1790209259889018 usec, L: 0
0924 00:21:10.173945 (+ 342us) write_op.cc:270] Start()
0924 00:21:10.174006 (+ 61us) write_op.cc:276] Timestamp: P: 1790209259894760 usec, L: 0
0924 00:21:10.174366 (+ 360us) write_op.cc:270] Start()
0924 00:21:10.174426 (+ 60us) write_op.cc:276] Timestamp: P: 1790209259929866 usec, L: 0
0924 00:21:10.174801 (+ 375us) write_op.cc:270] Start()
0924 00:21:10.174865 (+ 64us) write_op.cc:276] Timestamp: P: 1790209259969043 usec, L: 0
0924 00:21:10.175202 (+ 337us) write_op.cc:270] Start()
0924 00:21:10.175265 (+ 63us) write_op.cc:276] Timestamp: P: 1790209260011377 usec, L: 0
0924 00:21:10.175598 (+ 333us) write_op.cc:270] Start()
0924 00:21:10.175659 (+ 61us) write_op.cc:276] Timestamp: P: 1790209260033710 usec, L: 0
0924 00:21:10.176050 (+ 391us) write_op.cc:270] Start()
0924 00:21:10.176119 (+ 69us) write_op.cc:276] Timestamp: P: 1790209260085233 usec, L: 0
0924 00:21:10.176494 (+ 375us) write_op.cc:270] Start()
0924 00:21:10.176557 (+ 63us) write_op.cc:276] Timestamp: P: 1790209260095152 usec, L: 0
0924 00:21:10.176933 (+ 376us) write_op.cc:270] Start()
0924 00:21:10.177032 (+ 99us) write_op.cc:276] Timestamp: P: 1790209260204648 usec, L: 0
0924 00:21:10.188703 (+ 11671us) write_op.cc:270] Start()
0924 00:21:10.188781 (+ 78us) write_op.cc:276] Timestamp: P: 1790209260251718 usec, L: 0
0924 00:21:10.189303 (+ 522us) write_op.cc:270] Start()
0924 00:21:10.189368 (+ 65us) write_op.cc:276] Timestamp: P: 1790209260434617 usec, L: 0
0924 00:21:10.189741 (+ 373us) write_op.cc:270] Start()
0924 00:21:10.189800 (+ 59us) write_op.cc:276] Timestamp: P: 1790209260469150 usec, L: 0
0924 00:21:10.190183 (+ 383us) write_op.cc:270] Start()
0924 00:21:10.190245 (+ 62us) write_op.cc:276] Timestamp: P: 1790209260511570 usec, L: 0
0924 00:21:10.190705 (+ 460us) write_op.cc:270] Start()
0924 00:21:10.190769 (+ 64us) write_op.cc:276] Timestamp: P: 1790209260604999 usec, L: 0
0924 00:21:10.191216 (+ 447us) write_op.cc:270] Start()
0924 00:21:10.340818 (+149602us) write_op.cc:276] Timestamp: P: 1790209260676926 usec, L: 0
0924 00:21:10.341407 (+ 589us) write_op.cc:270] Start()
0924 00:21:10.341485 (+ 78us) write_op.cc:276] Timestamp: P: 1790209260760704 usec, L: 0
0924 00:21:10.341909 (+ 424us) write_op.cc:270] Start()
0924 00:21:10.341973 (+ 64us) write_op.cc:276] Timestamp: P: 1790209260805351 usec, L: 0
0924 00:21:10.342453 (+ 480us) write_op.cc:270] Start()
0924 00:21:10.342525 (+ 72us) write_op.cc:276] Timestamp: P: 1790209260853091 usec, L: 0
0924 00:21:10.342955 (+ 430us) write_op.cc:270] Start()
0924 00:21:10.343021 (+ 66us) write_op.cc:276] Timestamp: P: 1790209260901458 usec, L: 0
0924 00:21:10.343528 (+ 507us) write_op.cc:270] Start()
0924 00:21:10.343594 (+ 66us) write_op.cc:276] Timestamp: P: 1790209260964444 usec, L: 0
0924 00:21:10.343977 (+ 383us) write_op.cc:270] Start()
0924 00:21:10.344037 (+ 60us) write_op.cc:276] Timestamp: P: 1790209260967445 usec, L: 0
0924 00:21:10.344409 (+ 372us) write_op.cc:270] Start()
0924 00:21:10.344471 (+ 62us) write_op.cc:276] Timestamp: P: 1790209261010427 usec, L: 0
0924 00:21:10.344881 (+ 410us) write_op.cc:270] Start()
0924 00:21:10.344969 (+ 88us) write_op.cc:276] Timestamp: P: 1790209261032147 usec, L: 0
0924 00:21:10.345354 (+ 385us) write_op.cc:270] Start()
0924 00:21:10.345419 (+ 65us) write_op.cc:276] Timestamp: P: 1790209261066772 usec, L: 0
0924 00:21:10.345799 (+ 380us) write_op.cc:270] Start()
0924 00:21:10.345867 (+ 68us) write_op.cc:276] Timestamp: P: 1790209261096229 usec, L: 0
0924 00:21:10.346254 (+ 387us) write_op.cc:270] Start()
0924 00:21:10.346322 (+ 68us) write_op.cc:276] Timestamp: P: 1790209261149840 usec, L: 0
0924 00:21:10.346673 (+ 351us) write_op.cc:270] Start()
0924 00:21:10.346734 (+ 61us) write_op.cc:276] Timestamp: P: 1790209261196741 usec, L: 0
0924 00:21:10.347119 (+ 385us) write_op.cc:270] Start()
0924 00:21:10.347192 (+ 73us) write_op.cc:276] Timestamp: P: 1790209261258721 usec, L: 0
0924 00:21:10.347579 (+ 387us) write_op.cc:270] Start()
0924 00:21:10.347645 (+ 66us) write_op.cc:276] Timestamp: P: 1790209261304198 usec, L: 0
0924 00:21:10.348024 (+ 379us) write_op.cc:270] Start()
0924 00:21:10.348087 (+ 63us) write_op.cc:276] Timestamp: P: 1790209261347723 usec, L: 0
0924 00:21:10.348447 (+ 360us) write_op.cc:270] Start()
0924 00:21:10.348510 (+ 63us) write_op.cc:276] Timestamp: P: 1790209261363579 usec, L: 0
0924 00:21:10.348888 (+ 378us) write_op.cc:270] Start()
0924 00:21:10.412788 (+ 63900us) write_op.cc:276] Timestamp: P: 1790209261501125 usec, L: 0
0924 00:21:10.413413 (+ 625us) write_op.cc:270] Start()
0924 00:21:10.413488 (+ 75us) write_op.cc:276] Timestamp: P: 1790209261566116 usec, L: 0
0924 00:21:10.414015 (+ 527us) write_op.cc:270] Start()
0924 00:21:10.414086 (+ 71us) write_op.cc:276] Timestamp: P: 1790209261568223 usec, L: 0
0924 00:21:10.414480 (+ 394us) write_op.cc:270] Start()
0924 00:21:10.414546 (+ 66us) write_op.cc:276] Timestamp: P: 1790209261618551 usec, L: 0
0924 00:21:10.414994 (+ 448us) write_op.cc:270] Start()
0924 00:21:10.415062 (+ 68us) write_op.cc:276] Timestamp: P: 1790209261669721 usec, L: 0
0924 00:21:10.415429 (+ 367us) write_op.cc:270] Start()
0924 00:21:10.415490 (+ 61us) write_op.cc:276] Timestamp: P: 1790209261690466 usec, L: 0
0924 00:21:10.415855 (+ 365us) write_op.cc:270] Start()
0924 00:21:10.415918 (+ 63us) write_op.cc:276] Timestamp: P: 1790209261726569 usec, L: 0
0924 00:21:10.416276 (+ 358us) write_op.cc:270] Start()
0924 00:21:10.416347 (+ 71us) write_op.cc:276] Timestamp: P: 1790209261786597 usec, L: 0
0924 00:21:10.416701 (+ 354us) write_op.cc:270] Start()
0924 00:21:10.416763 (+ 62us) write_op.cc:276] Timestamp: P: 1790209261825864 usec, L: 0
0924 00:21:10.440202 (+ 23439us) write_op.cc:270] Start()
0924 00:21:10.440299 (+ 97us) write_op.cc:276] Timestamp: P: 1790209261833198 usec, L: 0
0924 00:21:10.440740 (+ 441us) write_op.cc:270] Start()
0924 00:21:10.440804 (+ 64us) write_op.cc:276] Timestamp: P: 1790209261891535 usec, L: 0
0924 00:21:10.441220 (+ 416us) write_op.cc:270] Start()
0924 00:21:10.441294 (+ 74us) write_op.cc:276] Timestamp: P: 1790209261932808 usec, L: 0
0924 00:21:10.441654 (+ 360us) write_op.cc:270] Start()
0924 00:21:10.441716 (+ 62us) write_op.cc:276] Timestamp: P: 1790209261939422 usec, L: 0
0924 00:21:10.472218 (+ 30502us) write_op.cc:270] Start()
0924 00:21:10.472307 (+ 89us) write_op.cc:276] Timestamp: P: 1790209261979578 usec, L: 0
0924 00:21:10.472715 (+ 408us) write_op.cc:270] Start()
0924 00:21:10.472776 (+ 61us) write_op.cc:276] Timestamp: P: 1790209262016739 usec, L: 0
0924 00:21:10.473143 (+ 367us) write_op.cc:270] Start()
0924 00:21:10.473204 (+ 61us) write_op.cc:276] Timestamp: P: 1790209262030536 usec, L: 0
0924 00:21:10.473569 (+ 365us) write_op.cc:270] Start()
0924 00:21:10.473634 (+ 65us) write_op.cc:276] Timestamp: P: 1790209262061148 usec, L: 0
0924 00:21:10.473959 (+ 325us) write_op.cc:270] Start()
0924 00:21:10.474019 (+ 60us) write_op.cc:276] Timestamp: P: 1790209262155938 usec, L: 0
0924 00:21:10.474372 (+ 353us) write_op.cc:270] Start()
0924 00:21:10.474432 (+ 60us) write_op.cc:276] Timestamp: P: 1790209262157954 usec, L: 0
0924 00:21:10.474770 (+ 338us) write_op.cc:270] Start()
0924 00:21:10.474832 (+ 62us) write_op.cc:276] Timestamp: P: 1790209262280086 usec, L: 0
0924 00:21:10.475159 (+ 327us) write_op.cc:270] Start()
0924 00:21:10.516908 (+ 41749us) write_op.cc:276] Timestamp: P: 1790209262338949 usec, L: 0
0924 00:21:10.517427 (+ 519us) write_op.cc:270] Start()
0924 00:21:10.517509 (+ 82us) write_op.cc:276] Timestamp: P: 1790209262431381 usec, L: 0
0924 00:21:10.517955 (+ 446us) write_op.cc:270] Start()
0924 00:21:10.518023 (+ 68us) write_op.cc:276] Timestamp: P: 1790209262469140 usec, L: 0
0924 00:21:10.518401 (+ 378us) write_op.cc:270] Start()
0924 00:21:10.518465 (+ 64us) write_op.cc:276] Timestamp: P: 1790209262571089 usec, L: 0
0924 00:21:10.518853 (+ 388us) write_op.cc:270] Start()
0924 00:21:10.518922 (+ 69us) write_op.cc:276] Timestamp: P: 1790209262624876 usec, L: 0
0924 00:21:10.519314 (+ 392us) write_op.cc:270] Start()
0924 00:21:10.519378 (+ 64us) write_op.cc:276] Timestamp: P: 1790209262644876 usec, L: 0
0924 00:21:10.519780 (+ 402us) write_op.cc:270] Start()
0924 00:21:10.519854 (+ 74us) write_op.cc:276] Timestamp: P: 1790209262676203 usec, L: 0
0924 00:21:10.520221 (+ 367us) write_op.cc:270] Start()
0924 00:21:10.520296 (+ 75us) write_op.cc:276] Timestamp: P: 1790209262687432 usec, L: 0
0924 00:21:10.520668 (+ 372us) write_op.cc:270] Start()
0924 00:21:10.520736 (+ 68us) write_op.cc:276] Timestamp: P: 1790209262733044 usec, L: 0
0924 00:21:10.557256 (+ 36520us) write_op.cc:270] Start()
0924 00:21:10.557346 (+ 90us) write_op.cc:276] Timestamp: P: 1790209262752078 usec, L: 0
0924 00:21:10.557879 (+ 533us) write_op.cc:270] Start()
0924 00:21:10.557948 (+ 69us) write_op.cc:276] Timestamp: P: 1790209262757660 usec, L: 0
0924 00:21:10.558335 (+ 387us) write_op.cc:270] Start()
0924 00:21:10.558400 (+ 65us) write_op.cc:276] Timestamp: P: 1790209262769341 usec, L: 0
0924 00:21:10.558780 (+ 380us) write_op.cc:270] Start()
0924 00:21:10.558858 (+ 78us) write_op.cc:276] Timestamp: P: 1790209262773997 usec, L: 0
0924 00:21:10.559209 (+ 351us) write_op.cc:270] Start()
0924 00:21:10.559273 (+ 64us) write_op.cc:276] Timestamp: P: 1790209262870994 usec, L: 0
0924 00:21:10.559638 (+ 365us) write_op.cc:270] Start()
0924 00:21:10.559699 (+ 61us) write_op.cc:276] Timestamp: P: 17902092
I20260924 00:21:18.641032 11979 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1d68651913494ce8afee3c136b2da9cc)
I20260924 00:21:18.641707 11979 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8bd0b3184f84f5f8da10c9c21472f92 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d68651913494ce8afee3c136b2da9cc" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 34479 } } peers { permanent_uuid: "b5aa3debe1cd45e19a6bbd0070913ac3" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 34487 } } peers { permanent_uuid: "f8bd0b3184f84f5f8da10c9c21472f92" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 35345 } }
W20260924 00:21:18.643572 11406 connection.cc:535] server connection from 127.0.0.1:59400 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260924 00:21:18.644003 10257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu with pid 11071
2026-09-24T00:21:18Z chronyd exiting
I20260924 00:21:18.694195 10257 master-stress-test.cc:586] Tables created: 185
I20260924 00:21:18.694399 10257 master-stress-test.cc:587] Tables altered: 93
I20260924 00:21:18.694543 10257 master-stress-test.cc:588] Tables deleted: 67
I20260924 00:21:18.694696 10257 master-stress-test.cc:589] Tablets replaced: 40
I20260924 00:21:18.694834 10257 master-stress-test.cc:590] Masters restarted: 1
[ OK ] CatalogManagerConfigurations/MasterStressTest.Test/0 (35751 ms)
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/3
I20260924 00:21:18.760354 10257 test_util.cc:268] Using random seed: -1809689205
2026-09-24T00:21:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:21:18Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskko7HPF/thirdparty/src/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskko7HPF/thirdparty/src/hadoop-3.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/security.properties
2026-09-24 00:21:25: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskko7HPF/thirdparty/src/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskko7HPF/thirdparty/src/hadoop-3.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
I20260924 00:21:33.012018 10257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu
/tmp/dist-test-taskko7HPF/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.4.126:37913
--webserver_interface=127.10.4.126
--webserver_port=0
--builtin_ntp_servers=127.10.4.84:44801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.4.126:37913,127.10.4.125:46875,127.10.4.124:45407
--hive_metastore_uris=thrift://127.0.0.1:45439
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260924 00:21:33.300623 12218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:21:33.301224 12218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:21:33.301666 12218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:21:33.332531 12218 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260924 00:21:33.332813 12218 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260924 00:21:33.333086 12218 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:21:33.333316 12218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:21:33.333639 12218 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:21:33.333945 12218 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:21:33.369513 12218 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.4.84:44801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:45439
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.10.4.126:37913,127.10.4.125:46875,127.10.4.124:45407
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.4.126:37913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.4.126
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:32:10 UTC on ccycloud.quasar-wfxeao.root.comops.site
build id 4668917
TSAN enabled
I20260924 00:21:33.370770 12218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:21:33.372370 12218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:21:33.389191 12225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:21:33.389201 12227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:21:33.389343 12224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:21:33.391662 12218 server_base.cc:1034] running on GCE node
I20260924 00:21:34.488260 12218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:21:34.490844 12218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:21:34.492223 12218 hybrid_clock.cc:648] HybridClock initialized: now 1790209294492193 us; error 41 us; skew 500 ppm
I20260924 00:21:34.493150 12218 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 00:21:34.499409 12218 webserver.cc:460] Webserver started at http://127.10.4.126:36309/ using document root <none> and password file <none>
I20260924 00:21:34.500308 12218 fs_manager.cc:359] Metadata directory not provided
I20260924 00:21:34.500504 12218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:21:34.501008 12218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:21:34.505735 12218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-0/data/instance:
uuid: "3c3a72a9440740a4948a000656ccabdf"
format_stamp: "Formatted at 2026-09-24 00:21:34 on dist-test-slave-g3nd"
I20260924 00:21:34.506819 12218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-0/wal/instance:
uuid: "3c3a72a9440740a4948a000656ccabdf"
format_stamp: "Formatted at 2026-09-24 00:21:34 on dist-test-slave-g3nd"
I20260924 00:21:34.513724 12218 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260924 00:21:34.519148 12235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:21:34.520293 12218 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260924 00:21:34.520617 12218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-0/data,/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-0/wal
uuid: "3c3a72a9440740a4948a000656ccabdf"
format_stamp: "Formatted at 2026-09-24 00:21:34 on dist-test-slave-g3nd"
I20260924 00:21:34.520967 12218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:21:34.552532 12218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:21:34.554006 12218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:21:34.554438 12218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:21:34.620402 12218 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.126:37913
I20260924 00:21:34.620507 12286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.4.126:37913 every 8 connection(s)
I20260924 00:21:34.623030 12218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-0/data/info.pb
I20260924 00:21:34.627995 12287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:34.628109 10257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu as pid 12218
I20260924 00:21:34.628777 10257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-0/wal/instance
I20260924 00:21:34.632424 10257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu
/tmp/dist-test-taskko7HPF/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.4.125:46875
--webserver_interface=127.10.4.125
--webserver_port=0
--builtin_ntp_servers=127.10.4.84:44801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.4.126:37913,127.10.4.125:46875,127.10.4.124:45407
--hive_metastore_uris=thrift://127.0.0.1:45439
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260924 00:21:34.642864 12287 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:21:34.670111 12287 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } has no permanent_uuid. Determining permanent_uuid...
W20260924 00:21:34.672638 12236 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
W20260924 00:21:34.676331 12287 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.125:46875: Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
I20260924 00:21:34.725803 12287 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } attempt: 1
W20260924 00:21:34.729511 12287 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.125:46875: Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
I20260924 00:21:34.797870 12287 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } attempt: 2
W20260924 00:21:34.801774 12287 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.125:46875: Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
I20260924 00:21:34.893136 12287 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } attempt: 3
W20260924 00:21:34.897212 12287 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.125:46875: Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
W20260924 00:21:34.930049 12288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:21:34.930626 12288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:21:34.931058 12288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:21:34.962287 12288 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260924 00:21:34.962564 12288 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260924 00:21:34.962811 12288 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:21:34.963049 12288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:21:34.963280 12288 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:21:34.963508 12288 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:21:34.998520 12288 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.4.84:44801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:45439
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.10.4.126:37913,127.10.4.125:46875,127.10.4.124:45407
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.4.125:46875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.4.125
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:32:10 UTC on ccycloud.quasar-wfxeao.root.comops.site
build id 4668917
TSAN enabled
I20260924 00:21:34.999729 12288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:21:35.001335 12288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:21:35.017948 12298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:21:35.040685 12287 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } attempt: 4
W20260924 00:21:35.044476 12287 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.125:46875: Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
I20260924 00:21:35.343892 12287 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } attempt: 5
W20260924 00:21:35.347327 12287 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.125:46875: Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
I20260924 00:21:35.894953 12287 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } attempt: 6
W20260924 00:21:35.903338 12287 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.125:46875: Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
W20260924 00:21:35.018116 12297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:21:35.017964 12300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:21:35.019654 12288 server_base.cc:1034] running on GCE node
I20260924 00:21:36.126091 12288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:21:36.128602 12288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:21:36.129979 12288 hybrid_clock.cc:648] HybridClock initialized: now 1790209296129928 us; error 55 us; skew 500 ppm
I20260924 00:21:36.130837 12288 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 00:21:36.137226 12288 webserver.cc:460] Webserver started at http://127.10.4.125:39815/ using document root <none> and password file <none>
I20260924 00:21:36.138235 12288 fs_manager.cc:359] Metadata directory not provided
I20260924 00:21:36.138479 12288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:21:36.138950 12288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:21:36.143836 12288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-1/data/instance:
uuid: "fb563a88aa00432ba12db382b1435a39"
format_stamp: "Formatted at 2026-09-24 00:21:36 on dist-test-slave-g3nd"
I20260924 00:21:36.145004 12288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-1/wal/instance:
uuid: "fb563a88aa00432ba12db382b1435a39"
format_stamp: "Formatted at 2026-09-24 00:21:36 on dist-test-slave-g3nd"
I20260924 00:21:36.152664 12288 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.004s
I20260924 00:21:36.158250 12309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:21:36.159389 12288 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260924 00:21:36.159713 12288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-1/data,/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-1/wal
uuid: "fb563a88aa00432ba12db382b1435a39"
format_stamp: "Formatted at 2026-09-24 00:21:36 on dist-test-slave-g3nd"
I20260924 00:21:36.160038 12288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:21:36.205016 12288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:21:36.206555 12288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:21:36.206981 12288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:21:36.273007 12288 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.125:46875
I20260924 00:21:36.273156 12360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.4.125:46875 every 8 connection(s)
I20260924 00:21:36.275636 12288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-1/data/info.pb
I20260924 00:21:36.278586 10257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu as pid 12288
I20260924 00:21:36.279080 10257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-1/wal/instance
I20260924 00:21:36.281421 12361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:36.282464 10257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu
/tmp/dist-test-taskko7HPF/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.4.124:45407
--webserver_interface=127.10.4.124
--webserver_port=0
--builtin_ntp_servers=127.10.4.84:44801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.4.126:37913,127.10.4.125:46875,127.10.4.124:45407
--hive_metastore_uris=thrift://127.0.0.1:45439
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260924 00:21:36.294685 12361 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:21:36.317394 12361 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:21:36.332523 12361 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } has no permanent_uuid. Determining permanent_uuid...
W20260924 00:21:36.334705 12313 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.124:45407: connect: Connection refused (error 111)
W20260924 00:21:36.338402 12361 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.124:45407: Network error: Client connection negotiation failed: client connection to 127.10.4.124:45407: connect: Connection refused (error 111)
I20260924 00:21:36.387956 12361 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } attempt: 1
W20260924 00:21:36.391567 12361 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.124:45407: Network error: Client connection negotiation failed: client connection to 127.10.4.124:45407: connect: Connection refused (error 111)
I20260924 00:21:36.460026 12361 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } attempt: 2
W20260924 00:21:36.463654 12361 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.124:45407: Network error: Client connection negotiation failed: client connection to 127.10.4.124:45407: connect: Connection refused (error 111)
I20260924 00:21:36.555123 12361 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } attempt: 3
W20260924 00:21:36.558981 12361 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.124:45407: Network error: Client connection negotiation failed: client connection to 127.10.4.124:45407: connect: Connection refused (error 111)
W20260924 00:21:36.578028 12362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:21:36.578640 12362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:21:36.579063 12362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:21:36.609702 12362 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260924 00:21:36.609972 12362 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260924 00:21:36.610255 12362 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:21:36.610488 12362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:21:36.610715 12362 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:21:36.610942 12362 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:21:36.646373 12362 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.4.84:44801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:45439
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.10.4.126:37913,127.10.4.125:46875,127.10.4.124:45407
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.4.124:45407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.4.124
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:32:10 UTC on ccycloud.quasar-wfxeao.root.comops.site
build id 4668917
TSAN enabled
I20260924 00:21:36.647591 12362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:21:36.649196 12362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:21:36.665714 12372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:21:36.702366 12361 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } attempt: 4
W20260924 00:21:36.706231 12361 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.124:45407: Network error: Client connection negotiation failed: client connection to 127.10.4.124:45407: connect: Connection refused (error 111)
I20260924 00:21:36.963832 12287 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } attempt: 7
I20260924 00:21:36.987910 12287 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } has no permanent_uuid. Determining permanent_uuid...
W20260924 00:21:36.992828 12287 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.124:45407: Network error: Client connection negotiation failed: client connection to 127.10.4.124:45407: connect: Connection refused (error 111)
I20260924 00:21:37.005581 12361 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } attempt: 5
W20260924 00:21:37.009513 12361 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.124:45407: Network error: Client connection negotiation failed: client connection to 127.10.4.124:45407: connect: Connection refused (error 111)
I20260924 00:21:37.051229 12287 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } attempt: 1
W20260924 00:21:37.054965 12287 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.124:45407: Network error: Client connection negotiation failed: client connection to 127.10.4.124:45407: connect: Connection refused (error 111)
I20260924 00:21:37.136408 12287 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } attempt: 2
W20260924 00:21:37.140244 12287 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.124:45407: Network error: Client connection negotiation failed: client connection to 127.10.4.124:45407: connect: Connection refused (error 111)
I20260924 00:21:37.225612 12287 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } attempt: 3
W20260924 00:21:37.229444 12287 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.124:45407: Network error: Client connection negotiation failed: client connection to 127.10.4.124:45407: connect: Connection refused (error 111)
I20260924 00:21:37.369786 12287 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } attempt: 4
W20260924 00:21:37.373147 12287 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.124:45407: Network error: Client connection negotiation failed: client connection to 127.10.4.124:45407: connect: Connection refused (error 111)
I20260924 00:21:37.556902 12361 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } attempt: 6
W20260924 00:21:37.563050 12361 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.124:45407: Network error: Client connection negotiation failed: client connection to 127.10.4.124:45407: connect: Connection refused (error 111)
I20260924 00:21:37.656685 12287 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } attempt: 5
W20260924 00:21:37.660303 12287 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.4.124:45407: Network error: Client connection negotiation failed: client connection to 127.10.4.124:45407: connect: Connection refused (error 111)
W20260924 00:21:36.665725 12373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:21:36.665755 12375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:21:36.667027 12362 server_base.cc:1034] running on GCE node
I20260924 00:21:37.772989 12362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:21:37.775471 12362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:21:37.776851 12362 hybrid_clock.cc:648] HybridClock initialized: now 1790209297776810 us; error 47 us; skew 500 ppm
I20260924 00:21:37.777733 12362 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 00:21:37.783697 12362 webserver.cc:460] Webserver started at http://127.10.4.124:36597/ using document root <none> and password file <none>
I20260924 00:21:37.784605 12362 fs_manager.cc:359] Metadata directory not provided
I20260924 00:21:37.784826 12362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:21:37.785295 12362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:21:37.790112 12362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/data/instance:
uuid: "dba2791dd9734eb4b84bbbf2f7d674b8"
format_stamp: "Formatted at 2026-09-24 00:21:37 on dist-test-slave-g3nd"
I20260924 00:21:37.791241 12362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/wal/instance:
uuid: "dba2791dd9734eb4b84bbbf2f7d674b8"
format_stamp: "Formatted at 2026-09-24 00:21:37 on dist-test-slave-g3nd"
I20260924 00:21:37.798249 12362 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260924 00:21:37.803745 12387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:21:37.804741 12362 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260924 00:21:37.805117 12362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/data,/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/wal
uuid: "dba2791dd9734eb4b84bbbf2f7d674b8"
format_stamp: "Formatted at 2026-09-24 00:21:37 on dist-test-slave-g3nd"
I20260924 00:21:37.805442 12362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:21:37.860285 12362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:21:37.861743 12362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:21:37.862165 12362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:21:37.926865 12362 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.124:45407
I20260924 00:21:37.926951 12438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.4.124:45407 every 8 connection(s)
I20260924 00:21:37.929486 12362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/data/info.pb
I20260924 00:21:37.934247 12439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:37.938467 10257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu as pid 12362
I20260924 00:21:37.938969 10257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/wal/instance
I20260924 00:21:37.948419 12439 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:21:37.979130 12439 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:21:37.990089 12439 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:21:38.008162 12439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8: Bootstrap starting.
I20260924 00:21:38.013751 12439 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:38.015854 12439 log.cc:824] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8: Log is configured to *not* fsync() on all Append() calls
I20260924 00:21:38.020817 12439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8: No bootstrap required, opened a new log
I20260924 00:21:38.041716 12439 raft_consensus.cc:348] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:38.042335 12439 raft_consensus.cc:374] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:38.042539 12439 raft_consensus.cc:729] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dba2791dd9734eb4b84bbbf2f7d674b8, State: Initialized, Role: FOLLOWER
I20260924 00:21:38.043169 12439 consensus_queue.cc:260] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:38.045377 12447 sys_catalog.cc:455] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } } }
I20260924 00:21:38.046008 12447 sys_catalog.cc:458] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:21:38.046680 12439 sys_catalog.cc:564] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [sys.catalog]: configured and running, proceeding with master startup.
W20260924 00:21:38.076599 12461 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 00:21:38.077124 12461 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260924 00:21:38.091363 12461 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 00:21:38.091804 12461 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 00:21:38.071699 12362 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: OK, instance_id { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" instance_seqno: 1790209294602562 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 00:21:38.102936 12362 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260924 00:21:38.114382 12461 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 00:21:38.117493 12393 tablet.cc:2369] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 00:21:38.119263 12461 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260924 00:21:38.134586 12461 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 00:21:38.134974 12461 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260924 00:21:38.149156 12461 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 00:21:38.149555 12461 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260924 00:21:38.163791 12461 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 00:21:38.164176 12461 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 00:21:38.174261 12447 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:38.174724 12447 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:38.177420 12447 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:21:38.183760 12461 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 00:21:38.184098 12461 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260924 00:21:38.198182 12461 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 00:21:38.198477 12461 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 00:21:38.212718 12287 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } attempt: 6
W20260924 00:21:38.215418 12461 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 00:21:38.215741 12461 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260924 00:21:38.230399 12461 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 00:21:38.230729 12461 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260924 00:21:38.244719 12461 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 00:21:38.245067 12461 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 00:21:38.245313 12287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf: Bootstrap starting.
I20260924 00:21:38.250523 12287 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:38.252265 12287 log.cc:824] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf: Log is configured to *not* fsync() on all Append() calls
I20260924 00:21:38.257189 12287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf: No bootstrap required, opened a new log
W20260924 00:21:38.259225 12461 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 00:21:38.259528 12461 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260924 00:21:38.273741 12461 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 00:21:38.274078 12461 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 00:21:38.285845 12287 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:38.289428 12287 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:38.289808 12287 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3a72a9440740a4948a000656ccabdf, State: Initialized, Role: FOLLOWER
I20260924 00:21:38.290817 12287 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
W20260924 00:21:38.292732 12461 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 00:21:38.293179 12461 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 00:21:38.303318 12466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } } }
I20260924 00:21:38.305210 12287 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [sys.catalog]: configured and running, proceeding with master startup.
W20260924 00:21:38.306747 12461 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260924 00:21:38.306856 12466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [sys.catalog]: This master's current role is: FOLLOWER
W20260924 00:21:38.307021 12461 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 00:21:38.307489 12262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3a72a9440740a4948a000656ccabdf" is_pre_election: true
I20260924 00:21:38.308420 12262 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dba2791dd9734eb4b84bbbf2f7d674b8 in term 0.
I20260924 00:21:38.309981 12391 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3a72a9440740a4948a000656ccabdf, dba2791dd9734eb4b84bbbf2f7d674b8; no voters:
I20260924 00:21:38.311024 12447 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:38.311359 12447 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:38.311654 12447 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 0 FOLLOWER]: Advancing to term 1
W20260924 00:21:38.313277 12241 tablet.cc:2369] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:21:38.322507 12447 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:38.325961 12261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3a72a9440740a4948a000656ccabdf"
I20260924 00:21:38.326664 12261 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:38.333705 12261 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dba2791dd9734eb4b84bbbf2f7d674b8 in term 1.
I20260924 00:21:38.334992 12391 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3a72a9440740a4948a000656ccabdf, dba2791dd9734eb4b84bbbf2f7d674b8; no voters:
I20260924 00:21:38.340224 12447 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
I20260924 00:21:38.341554 12447 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:38.342561 12447 raft_consensus.cc:686] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 1 LEADER]: Becoming Leader. State: Replica: dba2791dd9734eb4b84bbbf2f7d674b8, State: Running, Role: LEADER
I20260924 00:21:38.344978 12447 consensus_queue.cc:237] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
W20260924 00:21:38.359404 12461 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 00:21:38.361822 12461 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 00:21:38.360981 12469 sys_catalog.cc:455] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dba2791dd9734eb4b84bbbf2f7d674b8. Latest consensus state: current_term: 1 leader_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } } }
I20260924 00:21:38.362334 12469 sys_catalog.cc:458] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [sys.catalog]: This master's current role is: LEADER
I20260924 00:21:38.366228 12474 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:21:38.373750 12474 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:21:38.399590 12262 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 1 FOLLOWER]: Refusing update from remote peer dba2791dd9734eb4b84bbbf2f7d674b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:21:38.401753 12469 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:38.445952 12466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [sys.catalog]: SysCatalogTable state changed. Reason: New leader dba2791dd9734eb4b84bbbf2f7d674b8. Latest consensus state: current_term: 1 leader_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } } }
I20260924 00:21:38.454224 12466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [sys.catalog]: This master's current role is: FOLLOWER
W20260924 00:21:38.460223 12485 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 00:21:38.460546 12485 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 00:21:38.468065 12480 mvcc.cc:205] Tried to move back new op lower bound from 7332697286222143488 to 7332697286033956864. Current Snapshot: MvccSnapshot[applied={T|T < 7332697286222143488}]
I20260924 00:21:38.471439 12486 mvcc.cc:205] Tried to move back new op lower bound from 7332697286222143488 to 7332697286033956864. Current Snapshot: MvccSnapshot[applied={T|T < 7332697286222143488}]
I20260924 00:21:38.476379 12469 sys_catalog.cc:455] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } } }
I20260924 00:21:38.478652 12469 sys_catalog.cc:458] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [sys.catalog]: This master's current role is: LEADER
I20260924 00:21:38.489146 12474 catalog_manager.cc:1347] Generated new cluster ID: 575a504d93ea472a8cb263817ca44ba4
I20260924 00:21:38.489429 12474 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:21:38.445947 12218 master_runner.cc:186] Error getting master registration for 127.10.4.125:46875: OK, instance_id { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" instance_seqno: 1790209296254919 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 00:21:38.493173 12218 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:21:38.497557 12466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } } }
I20260924 00:21:38.498250 12466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:21:38.501510 12485 catalog_manager.cc:1259] Loaded cluster ID: 575a504d93ea472a8cb263817ca44ba4
I20260924 00:21:38.501746 12485 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf: loading cluster ID for follower catalog manager: success
W20260924 00:21:38.505709 12485 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260924 00:21:38.505981 12485 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
W20260924 00:21:38.521232 12485 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260924 00:21:38.521529 12485 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260924 00:21:38.525406 12474 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:21:38.527880 12474 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:21:38.548902 12485 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf: acquiring CA information for follower catalog manager: success
W20260924 00:21:38.554072 12485 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260924 00:21:38.560680 12474 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8: Generated new TSK 0
I20260924 00:21:38.562781 12474 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:21:38.577880 12474 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
W20260924 00:21:38.581552 12485 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260924 00:21:38.583113 12474 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260924 00:21:38.607225 12485 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 00:21:38.608664 10257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu
/tmp/dist-test-taskko7HPF/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.4.65:0
--local_ip_for_outbound_sockets=127.10.4.65
--webserver_interface=127.10.4.65
--webserver_port=0
--tserver_master_addrs=127.10.4.126:37913,127.10.4.125:46875,127.10.4.124:45407
--builtin_ntp_servers=127.10.4.84:44801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260924 00:21:38.623373 12361 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } attempt: 7
I20260924 00:21:38.667397 12361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39: Bootstrap starting.
I20260924 00:21:38.674036 12361 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:38.687274 12361 log.cc:824] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39: Log is configured to *not* fsync() on all Append() calls
W20260924 00:21:38.698081 12388 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.4.125:46875 timed out after 0.519s (SENT)
I20260924 00:21:38.702641 12361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39: No bootstrap required, opened a new log
I20260924 00:21:38.762396 12361 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:38.763429 12361 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:38.763782 12361 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb563a88aa00432ba12db382b1435a39, State: Initialized, Role: FOLLOWER
I20260924 00:21:38.764853 12361 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:38.774461 12495 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } } }
I20260924 00:21:38.776711 12495 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:21:38.779181 12361 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:21:38.779340 12336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb563a88aa00432ba12db382b1435a39" is_pre_election: true
I20260924 00:21:38.780216 12336 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dba2791dd9734eb4b84bbbf2f7d674b8 in term 0.
W20260924 00:21:38.780773 12336 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39334 (request call id 1) took 602 ms (client timeout 519 ms). Trace:
W20260924 00:21:38.781157 12336 rpcz_store.cc:269] 0924 00:21:38.178338 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:38.178491 (+ 153us) service_pool.cc:227] Handling call
0924 00:21:38.780695 (+602204us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 00:21:38.782583 12333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb563a88aa00432ba12db382b1435a39" is_pre_election: true
I20260924 00:21:38.783228 12333 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dba2791dd9734eb4b84bbbf2f7d674b8 in term 0.
I20260924 00:21:38.784260 12334 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:38.789203 12335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb563a88aa00432ba12db382b1435a39"
W20260924 00:21:38.790141 12335 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39334 (request call id 2) took 449 ms (client timeout 510 ms). Trace:
W20260924 00:21:38.790444 12335 rpcz_store.cc:269] 0924 00:21:38.340929 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:38.341113 (+ 184us) service_pool.cc:227] Handling call
0924 00:21:38.790086 (+448973us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 00:21:38.795033 12334 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [term 1 FOLLOWER]: Refusing update from remote peer dba2791dd9734eb4b84bbbf2f7d674b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:21:38.796648 12447 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:38.869282 12495 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dba2791dd9734eb4b84bbbf2f7d674b8. Latest consensus state: current_term: 1 leader_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } } }
I20260924 00:21:38.870072 12495 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:21:38.871709 12505 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } } }
I20260924 00:21:38.872563 12505 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:21:38.949327 12515 catalog_manager.cc:1259] Loaded cluster ID: 575a504d93ea472a8cb263817ca44ba4
I20260924 00:21:38.957999 12515 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39: loading cluster ID for follower catalog manager: success
I20260924 00:21:38.965785 12515 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39: acquiring CA information for follower catalog manager: success
I20260924 00:21:38.979468 12515 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 00:21:39.082930 12517 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dba2791dd9734eb4b84bbbf2f7d674b8)
I20260924 00:21:39.083608 12517 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:39.086802 12335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3c3a72a9440740a4948a000656ccabdf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fb563a88aa00432ba12db382b1435a39" is_pre_election: true
I20260924 00:21:39.091217 12414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3c3a72a9440740a4948a000656ccabdf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" is_pre_election: true
I20260924 00:21:39.093462 12239 leader_election.cc:302] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3a72a9440740a4948a000656ccabdf; no voters: dba2791dd9734eb4b84bbbf2f7d674b8, fb563a88aa00432ba12db382b1435a39
I20260924 00:21:39.096216 12517 leader_election.cc:288] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875), dba2791dd9734eb4b84bbbf2f7d674b8 (127.10.4.124:45407)
I20260924 00:21:39.097162 12517 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260924 00:21:39.216864 12491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:21:39.217739 12491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:21:39.218729 12491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:21:39.291587 12491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:21:39.297235 12491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.4.65
I20260924 00:21:39.301851 12505 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dba2791dd9734eb4b84bbbf2f7d674b8)
I20260924 00:21:39.302323 12505 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:39.304463 12505 leader_election.cc:288] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), dba2791dd9734eb4b84bbbf2f7d674b8 (127.10.4.124:45407)
I20260924 00:21:39.305896 12262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fb563a88aa00432ba12db382b1435a39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3a72a9440740a4948a000656ccabdf" is_pre_election: true
I20260924 00:21:39.307090 12414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fb563a88aa00432ba12db382b1435a39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" is_pre_election: true
I20260924 00:21:39.307878 12447 sys_catalog.cc:455] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } } }
I20260924 00:21:39.308598 12447 sys_catalog.cc:458] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [sys.catalog]: This master's current role is: LEADER
I20260924 00:21:39.308853 12313 leader_election.cc:302] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb563a88aa00432ba12db382b1435a39; no voters: 3c3a72a9440740a4948a000656ccabdf, dba2791dd9734eb4b84bbbf2f7d674b8
I20260924 00:21:39.310038 12505 raft_consensus.cc:2726] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 00:21:39.419680 12491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.4.84:44801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.4.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.4.65
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.10.4.126:37913,127.10.4.125:46875,127.10.4.124:45407
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.4.65
--log_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:32:10 UTC on ccycloud.quasar-wfxeao.root.comops.site
build id 4668917
TSAN enabled
I20260924 00:21:39.421592 12491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:21:39.424000 12491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:21:39.466902 12527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:21:39.660071 12530 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dba2791dd9734eb4b84bbbf2f7d674b8)
I20260924 00:21:39.662672 12530 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:39.666659 12335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3c3a72a9440740a4948a000656ccabdf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fb563a88aa00432ba12db382b1435a39" is_pre_election: true
I20260924 00:21:39.667729 12414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3c3a72a9440740a4948a000656ccabdf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" is_pre_election: true
I20260924 00:21:39.669603 12239 leader_election.cc:302] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3a72a9440740a4948a000656ccabdf; no voters: dba2791dd9734eb4b84bbbf2f7d674b8, fb563a88aa00432ba12db382b1435a39
I20260924 00:21:39.681589 12530 leader_election.cc:288] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875), dba2791dd9734eb4b84bbbf2f7d674b8 (127.10.4.124:45407)
I20260924 00:21:39.683034 12530 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 00:21:40.653891 12532 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dba2791dd9734eb4b84bbbf2f7d674b8)
I20260924 00:21:40.654476 12532 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:40.656692 12532 leader_election.cc:288] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), dba2791dd9734eb4b84bbbf2f7d674b8 (127.10.4.124:45407)
I20260924 00:21:40.658506 12414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fb563a88aa00432ba12db382b1435a39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" is_pre_election: true
I20260924 00:21:40.658547 12262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fb563a88aa00432ba12db382b1435a39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3a72a9440740a4948a000656ccabdf" is_pre_election: true
I20260924 00:21:40.660411 12313 leader_election.cc:302] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb563a88aa00432ba12db382b1435a39; no voters: 3c3a72a9440740a4948a000656ccabdf, dba2791dd9734eb4b84bbbf2f7d674b8
I20260924 00:21:40.661172 12532 raft_consensus.cc:2726] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 00:21:41.192303 12533 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dba2791dd9734eb4b84bbbf2f7d674b8)
I20260924 00:21:41.193073 12533 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:41.195149 12533 leader_election.cc:288] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), dba2791dd9734eb4b84bbbf2f7d674b8 (127.10.4.124:45407)
I20260924 00:21:41.196542 12262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fb563a88aa00432ba12db382b1435a39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3a72a9440740a4948a000656ccabdf" is_pre_election: true
I20260924 00:21:41.199815 12414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fb563a88aa00432ba12db382b1435a39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" is_pre_election: true
I20260924 00:21:41.201624 12313 leader_election.cc:302] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb563a88aa00432ba12db382b1435a39; no voters: 3c3a72a9440740a4948a000656ccabdf, dba2791dd9734eb4b84bbbf2f7d674b8
I20260924 00:21:41.202805 12533 raft_consensus.cc:2726] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260924 00:21:40.846104 12525 debug-util.cc:398] Leaking SignalData structure 0x7b08000148c0 after lost signal to thread 12491
W20260924 00:21:41.798544 12525 kernel_stack_watchdog.cc:198] Thread 12491 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 382ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20260924 00:21:41.798691 12491 thread.cc:636] GCE (cloud detector) Time spent creating pthread: real 2.337s user 0.480s sys 0.845s
W20260924 00:21:41.799099 12491 thread.cc:602] GCE (cloud detector) Time spent starting thread: real 2.337s user 0.481s sys 0.846s
W20260924 00:21:39.469751 12526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:21:41.807189 12535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:21:41.807777 12491 server_base.cc:1034] running on GCE node
I20260924 00:21:41.808892 12491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:21:41.811285 12491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:21:41.812729 12491 hybrid_clock.cc:648] HybridClock initialized: now 1790209301812687 us; error 37 us; skew 500 ppm
I20260924 00:21:41.813597 12491 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 00:21:41.825201 12491 webserver.cc:460] Webserver started at http://127.10.4.65:33855/ using document root <none> and password file <none>
I20260924 00:21:41.826156 12491 fs_manager.cc:359] Metadata directory not provided
I20260924 00:21:41.826345 12491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:21:41.826766 12491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:21:41.831420 12491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-0/data/instance:
uuid: "29842918acc045d29ea0fa4adfff5b9c"
format_stamp: "Formatted at 2026-09-24 00:21:41 on dist-test-slave-g3nd"
I20260924 00:21:41.832598 12491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-0/wal/instance:
uuid: "29842918acc045d29ea0fa4adfff5b9c"
format_stamp: "Formatted at 2026-09-24 00:21:41 on dist-test-slave-g3nd"
I20260924 00:21:41.840134 12491 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260924 00:21:41.845947 12540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:21:41.847093 12491 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260924 00:21:41.847402 12491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-0/data,/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-0/wal
uuid: "29842918acc045d29ea0fa4adfff5b9c"
format_stamp: "Formatted at 2026-09-24 00:21:41 on dist-test-slave-g3nd"
I20260924 00:21:41.847707 12491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:21:41.900107 12491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:21:41.901786 12491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:21:41.902215 12491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:21:41.904640 12491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:21:41.908996 12491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:21:41.909235 12491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260924 00:21:41.909500 12491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:21:41.909646 12491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:21:42.014338 12627 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dba2791dd9734eb4b84bbbf2f7d674b8)
I20260924 00:21:42.014828 12627 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:42.017812 12627 leader_election.cc:288] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875), dba2791dd9734eb4b84bbbf2f7d674b8 (127.10.4.124:45407)
I20260924 00:21:42.017983 12414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3c3a72a9440740a4948a000656ccabdf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" is_pre_election: true
I20260924 00:21:42.018354 12335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3c3a72a9440740a4948a000656ccabdf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fb563a88aa00432ba12db382b1435a39" is_pre_election: true
I20260924 00:21:42.019613 12236 leader_election.cc:302] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3a72a9440740a4948a000656ccabdf; no voters: dba2791dd9734eb4b84bbbf2f7d674b8, fb563a88aa00432ba12db382b1435a39
I20260924 00:21:42.020218 12627 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 00:21:42.048321 12491 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.65:46421
I20260924 00:21:42.048460 12653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.4.65:46421 every 8 connection(s)
I20260924 00:21:42.051123 12491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-0/data/info.pb
I20260924 00:21:42.060863 10257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu as pid 12491
I20260924 00:21:42.061367 10257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-0/wal/instance
I20260924 00:21:42.072927 10257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu
/tmp/dist-test-taskko7HPF/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.4.66:0
--local_ip_for_outbound_sockets=127.10.4.66
--webserver_interface=127.10.4.66
--webserver_port=0
--tserver_master_addrs=127.10.4.126:37913,127.10.4.125:46875,127.10.4.124:45407
--builtin_ntp_servers=127.10.4.84:44801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260924 00:21:42.088692 12655 heartbeater.cc:347] Connected to a master server at 127.10.4.125:46875
I20260924 00:21:42.089210 12655 heartbeater.cc:464] Registering TS with master...
I20260924 00:21:42.090328 12655 heartbeater.cc:511] Master 127.10.4.125:46875 requested a full tablet report, sending...
I20260924 00:21:42.090824 12656 heartbeater.cc:347] Connected to a master server at 127.10.4.126:37913
I20260924 00:21:42.091147 12656 heartbeater.cc:464] Registering TS with master...
I20260924 00:21:42.091846 12656 heartbeater.cc:511] Master 127.10.4.126:37913 requested a full tablet report, sending...
I20260924 00:21:42.093297 12326 ts_manager.cc:194] Registered new tserver with Master: 29842918acc045d29ea0fa4adfff5b9c (127.10.4.65:46421)
I20260924 00:21:42.094749 12252 ts_manager.cc:194] Registered new tserver with Master: 29842918acc045d29ea0fa4adfff5b9c (127.10.4.65:46421)
I20260924 00:21:42.097254 12654 heartbeater.cc:347] Connected to a master server at 127.10.4.124:45407
I20260924 00:21:42.097536 12654 heartbeater.cc:464] Registering TS with master...
I20260924 00:21:42.098150 12654 heartbeater.cc:511] Master 127.10.4.124:45407 requested a full tablet report, sending...
I20260924 00:21:42.100065 12404 ts_manager.cc:194] Registered new tserver with Master: 29842918acc045d29ea0fa4adfff5b9c (127.10.4.65:46421)
I20260924 00:21:42.102079 12404 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.4.65:44137
W20260924 00:21:42.386917 12664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:21:42.387406 12664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:21:42.388073 12664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:21:42.420774 12664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:21:42.421762 12664 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.4.66
I20260924 00:21:42.457693 12664 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.4.84:44801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.4.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.4.66
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.10.4.126:37913,127.10.4.125:46875,127.10.4.124:45407
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.4.66
--log_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:32:10 UTC on ccycloud.quasar-wfxeao.root.comops.site
build id 4668917
TSAN enabled
I20260924 00:21:42.458920 12664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:21:42.460594 12664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:21:42.480000 12673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:21:42.493806 12676 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dba2791dd9734eb4b84bbbf2f7d674b8)
I20260924 00:21:42.494182 12676 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:42.495636 12676 leader_election.cc:288] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), dba2791dd9734eb4b84bbbf2f7d674b8 (127.10.4.124:45407)
I20260924 00:21:42.496490 12262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fb563a88aa00432ba12db382b1435a39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3a72a9440740a4948a000656ccabdf" is_pre_election: true
I20260924 00:21:42.496734 12414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fb563a88aa00432ba12db382b1435a39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" is_pre_election: true
I20260924 00:21:42.498135 12313 leader_election.cc:302] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb563a88aa00432ba12db382b1435a39; no voters: 3c3a72a9440740a4948a000656ccabdf, dba2791dd9734eb4b84bbbf2f7d674b8
I20260924 00:21:42.498706 12676 raft_consensus.cc:2726] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 00:21:42.606235 12654 heartbeater.cc:503] Master 127.10.4.124:45407 was elected leader, sending a full tablet report...
I20260924 00:21:42.613464 12677 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dba2791dd9734eb4b84bbbf2f7d674b8)
I20260924 00:21:42.613916 12677 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:42.615479 12677 leader_election.cc:288] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875), dba2791dd9734eb4b84bbbf2f7d674b8 (127.10.4.124:45407)
I20260924 00:21:42.616289 12335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3c3a72a9440740a4948a000656ccabdf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fb563a88aa00432ba12db382b1435a39" is_pre_election: true
I20260924 00:21:42.617354 12414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3c3a72a9440740a4948a000656ccabdf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" is_pre_election: true
I20260924 00:21:42.618527 12239 leader_election.cc:302] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3a72a9440740a4948a000656ccabdf; no voters: dba2791dd9734eb4b84bbbf2f7d674b8, fb563a88aa00432ba12db382b1435a39
I20260924 00:21:42.619136 12677 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 00:21:43.074319 12678 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dba2791dd9734eb4b84bbbf2f7d674b8)
I20260924 00:21:43.074705 12678 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:43.076159 12678 leader_election.cc:288] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), dba2791dd9734eb4b84bbbf2f7d674b8 (127.10.4.124:45407)
I20260924 00:21:43.077147 12262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fb563a88aa00432ba12db382b1435a39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3a72a9440740a4948a000656ccabdf" is_pre_election: true
I20260924 00:21:43.077392 12414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fb563a88aa00432ba12db382b1435a39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" is_pre_election: true
I20260924 00:21:43.078815 12313 leader_election.cc:302] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb563a88aa00432ba12db382b1435a39; no voters: 3c3a72a9440740a4948a000656ccabdf, dba2791dd9734eb4b84bbbf2f7d674b8
I20260924 00:21:43.079422 12678 raft_consensus.cc:2726] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260924 00:21:42.480522 12671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:21:42.482133 12664 server_base.cc:1034] running on GCE node
W20260924 00:21:42.481264 12670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:21:43.583441 12664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:21:43.585676 12664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:21:43.587069 12664 hybrid_clock.cc:648] HybridClock initialized: now 1790209303587024 us; error 53 us; skew 500 ppm
I20260924 00:21:43.587857 12664 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 00:21:43.594298 12664 webserver.cc:460] Webserver started at http://127.10.4.66:42775/ using document root <none> and password file <none>
I20260924 00:21:43.595197 12664 fs_manager.cc:359] Metadata directory not provided
I20260924 00:21:43.595404 12664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:21:43.595837 12664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:21:43.601387 12664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-1/data/instance:
uuid: "fa329f7568e1416597742422ff0026cb"
format_stamp: "Formatted at 2026-09-24 00:21:43 on dist-test-slave-g3nd"
I20260924 00:21:43.602643 12664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-1/wal/instance:
uuid: "fa329f7568e1416597742422ff0026cb"
format_stamp: "Formatted at 2026-09-24 00:21:43 on dist-test-slave-g3nd"
I20260924 00:21:43.611301 12664 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.005s sys 0.004s
I20260924 00:21:43.617487 12686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:21:43.618539 12664 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260924 00:21:43.618937 12664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-1/data,/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-1/wal
uuid: "fa329f7568e1416597742422ff0026cb"
format_stamp: "Formatted at 2026-09-24 00:21:43 on dist-test-slave-g3nd"
I20260924 00:21:43.619393 12664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:21:43.631068 12687 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dba2791dd9734eb4b84bbbf2f7d674b8)
I20260924 00:21:43.631583 12687 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:43.633383 12687 leader_election.cc:288] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875), dba2791dd9734eb4b84bbbf2f7d674b8 (127.10.4.124:45407)
I20260924 00:21:43.634485 12335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3c3a72a9440740a4948a000656ccabdf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fb563a88aa00432ba12db382b1435a39" is_pre_election: true
I20260924 00:21:43.634560 12414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3c3a72a9440740a4948a000656ccabdf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" is_pre_election: true
I20260924 00:21:43.636140 12236 leader_election.cc:302] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3a72a9440740a4948a000656ccabdf; no voters: dba2791dd9734eb4b84bbbf2f7d674b8, fb563a88aa00432ba12db382b1435a39
I20260924 00:21:43.636996 12687 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 00:21:43.688208 12664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:21:43.689682 12664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:21:43.690132 12664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:21:43.692507 12664 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:21:43.696444 12664 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:21:43.696632 12664 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:21:43.696884 12664 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:21:43.697055 12664 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260924 00:21:43.830261 12664 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.66:40645
I20260924 00:21:43.830392 12799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.4.66:40645 every 8 connection(s)
I20260924 00:21:43.833137 12664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-1/data/info.pb
I20260924 00:21:43.836917 10257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu as pid 12664
I20260924 00:21:43.837687 10257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-1/wal/instance
I20260924 00:21:43.854476 10257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu
/tmp/dist-test-taskko7HPF/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.4.67:0
--local_ip_for_outbound_sockets=127.10.4.67
--webserver_interface=127.10.4.67
--webserver_port=0
--tserver_master_addrs=127.10.4.126:37913,127.10.4.125:46875,127.10.4.124:45407
--builtin_ntp_servers=127.10.4.84:44801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260924 00:21:43.889556 12800 heartbeater.cc:347] Connected to a master server at 127.10.4.124:45407
I20260924 00:21:43.890039 12800 heartbeater.cc:464] Registering TS with master...
I20260924 00:21:43.891140 12800 heartbeater.cc:511] Master 127.10.4.124:45407 requested a full tablet report, sending...
I20260924 00:21:43.891433 12802 heartbeater.cc:347] Connected to a master server at 127.10.4.126:37913
I20260924 00:21:43.891741 12802 heartbeater.cc:464] Registering TS with master...
I20260924 00:21:43.892482 12802 heartbeater.cc:511] Master 127.10.4.126:37913 requested a full tablet report, sending...
I20260924 00:21:43.893152 12801 heartbeater.cc:347] Connected to a master server at 127.10.4.125:46875
I20260924 00:21:43.893478 12801 heartbeater.cc:464] Registering TS with master...
I20260924 00:21:43.893489 12404 ts_manager.cc:194] Registered new tserver with Master: fa329f7568e1416597742422ff0026cb (127.10.4.66:40645)
I20260924 00:21:43.894204 12801 heartbeater.cc:511] Master 127.10.4.125:46875 requested a full tablet report, sending...
I20260924 00:21:43.895236 12252 ts_manager.cc:194] Registered new tserver with Master: fa329f7568e1416597742422ff0026cb (127.10.4.66:40645)
I20260924 00:21:43.895330 12404 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.4.66:33045
I20260924 00:21:43.897406 12326 ts_manager.cc:194] Registered new tserver with Master: fa329f7568e1416597742422ff0026cb (127.10.4.66:40645)
I20260924 00:21:44.101037 12811 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dba2791dd9734eb4b84bbbf2f7d674b8)
I20260924 00:21:44.101447 12811 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:44.102891 12811 leader_election.cc:288] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), dba2791dd9734eb4b84bbbf2f7d674b8 (127.10.4.124:45407)
I20260924 00:21:44.104061 12414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fb563a88aa00432ba12db382b1435a39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" is_pre_election: true
I20260924 00:21:44.104158 12262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fb563a88aa00432ba12db382b1435a39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3a72a9440740a4948a000656ccabdf" is_pre_election: true
I20260924 00:21:44.105715 12313 leader_election.cc:302] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb563a88aa00432ba12db382b1435a39; no voters: 3c3a72a9440740a4948a000656ccabdf, dba2791dd9734eb4b84bbbf2f7d674b8
I20260924 00:21:44.106588 12811 raft_consensus.cc:2726] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260924 00:21:44.186916 12810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:21:44.187431 12810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:21:44.188025 12810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:21:44.221372 12810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:21:44.222342 12810 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.4.67
I20260924 00:21:44.240129 12816 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dba2791dd9734eb4b84bbbf2f7d674b8)
I20260924 00:21:44.240486 12816 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:44.241986 12816 leader_election.cc:288] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875), dba2791dd9734eb4b84bbbf2f7d674b8 (127.10.4.124:45407)
I20260924 00:21:44.242776 12335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3c3a72a9440740a4948a000656ccabdf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fb563a88aa00432ba12db382b1435a39" is_pre_election: true
I20260924 00:21:44.243122 12414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3c3a72a9440740a4948a000656ccabdf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" is_pre_election: true
I20260924 00:21:44.244369 12239 leader_election.cc:302] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3a72a9440740a4948a000656ccabdf; no voters: dba2791dd9734eb4b84bbbf2f7d674b8, fb563a88aa00432ba12db382b1435a39
I20260924 00:21:44.245060 12816 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 00:21:44.258500 12810 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.4.84:44801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.4.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.4.67
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.10.4.126:37913,127.10.4.125:46875,127.10.4.124:45407
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.4.67
--log_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:32:10 UTC on ccycloud.quasar-wfxeao.root.comops.site
build id 4668917
TSAN enabled
I20260924 00:21:44.259677 12810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:21:44.261270 12810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:21:44.281476 12818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:21:44.400303 12800 heartbeater.cc:503] Master 127.10.4.124:45407 was elected leader, sending a full tablet report...
I20260924 00:21:44.628091 12824 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dba2791dd9734eb4b84bbbf2f7d674b8)
I20260924 00:21:44.628509 12824 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:44.630152 12824 leader_election.cc:288] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), dba2791dd9734eb4b84bbbf2f7d674b8 (127.10.4.124:45407)
I20260924 00:21:44.631178 12262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fb563a88aa00432ba12db382b1435a39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3a72a9440740a4948a000656ccabdf" is_pre_election: true
I20260924 00:21:44.631336 12414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fb563a88aa00432ba12db382b1435a39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" is_pre_election: true
I20260924 00:21:44.632761 12313 leader_election.cc:302] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb563a88aa00432ba12db382b1435a39; no voters: 3c3a72a9440740a4948a000656ccabdf, dba2791dd9734eb4b84bbbf2f7d674b8
I20260924 00:21:44.633764 12824 raft_consensus.cc:2726] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 00:21:45.250792 12826 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dba2791dd9734eb4b84bbbf2f7d674b8)
I20260924 00:21:45.251204 12826 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:45.252707 12826 leader_election.cc:288] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875), dba2791dd9734eb4b84bbbf2f7d674b8 (127.10.4.124:45407)
I20260924 00:21:45.253821 12414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3c3a72a9440740a4948a000656ccabdf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" is_pre_election: true
I20260924 00:21:45.253830 12335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3c3a72a9440740a4948a000656ccabdf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fb563a88aa00432ba12db382b1435a39" is_pre_election: true
I20260924 00:21:45.255383 12236 leader_election.cc:302] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3a72a9440740a4948a000656ccabdf; no voters: dba2791dd9734eb4b84bbbf2f7d674b8, fb563a88aa00432ba12db382b1435a39
I20260924 00:21:45.255987 12826 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260924 00:21:44.282483 12819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:21:44.283546 12810 server_base.cc:1034] running on GCE node
W20260924 00:21:44.283716 12821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:21:45.407636 12810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:21:45.410322 12810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:21:45.411828 12810 hybrid_clock.cc:648] HybridClock initialized: now 1790209305411788 us; error 41 us; skew 500 ppm
I20260924 00:21:45.412781 12810 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 00:21:45.419149 12810 webserver.cc:460] Webserver started at http://127.10.4.67:32919/ using document root <none> and password file <none>
I20260924 00:21:45.420105 12810 fs_manager.cc:359] Metadata directory not provided
I20260924 00:21:45.420324 12810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:21:45.420763 12810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:21:45.425611 12810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-2/data/instance:
uuid: "08e4cf0195e645bcbb5ac5e3ede93096"
format_stamp: "Formatted at 2026-09-24 00:21:45 on dist-test-slave-g3nd"
I20260924 00:21:45.426669 12810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-2/wal/instance:
uuid: "08e4cf0195e645bcbb5ac5e3ede93096"
format_stamp: "Formatted at 2026-09-24 00:21:45 on dist-test-slave-g3nd"
I20260924 00:21:45.433905 12810 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260924 00:21:45.439661 12831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:21:45.440707 12810 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260924 00:21:45.441056 12810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-2/data,/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-2/wal
uuid: "08e4cf0195e645bcbb5ac5e3ede93096"
format_stamp: "Formatted at 2026-09-24 00:21:45 on dist-test-slave-g3nd"
I20260924 00:21:45.441385 12810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:21:45.478907 12810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:21:45.480391 12810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:21:45.480841 12810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:21:45.483366 12810 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:21:45.487536 12810 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:21:45.487720 12810 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:21:45.487964 12810 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:21:45.488099 12810 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:21:45.627180 12810 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.67:38139
I20260924 00:21:45.627290 12943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.4.67:38139 every 8 connection(s)
I20260924 00:21:45.630764 12810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-2/data/info.pb
I20260924 00:21:45.633961 10257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu as pid 12810
I20260924 00:21:45.635818 10257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-2/wal/instance
I20260924 00:21:45.696201 12944 heartbeater.cc:347] Connected to a master server at 127.10.4.124:45407
I20260924 00:21:45.696712 12944 heartbeater.cc:464] Registering TS with master...
I20260924 00:21:45.697793 12944 heartbeater.cc:511] Master 127.10.4.124:45407 requested a full tablet report, sending...
I20260924 00:21:45.700233 12404 ts_manager.cc:194] Registered new tserver with Master: 08e4cf0195e645bcbb5ac5e3ede93096 (127.10.4.67:38139)
I20260924 00:21:45.702152 12404 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.4.67:51021
I20260924 00:21:45.709779 12948 heartbeater.cc:347] Connected to a master server at 127.10.4.126:37913
I20260924 00:21:45.710101 12948 heartbeater.cc:464] Registering TS with master...
I20260924 00:21:45.710744 12948 heartbeater.cc:511] Master 127.10.4.126:37913 requested a full tablet report, sending...
I20260924 00:21:45.713419 12252 ts_manager.cc:194] Registered new tserver with Master: 08e4cf0195e645bcbb5ac5e3ede93096 (127.10.4.67:38139)
I20260924 00:21:45.713922 12945 heartbeater.cc:347] Connected to a master server at 127.10.4.125:46875
I20260924 00:21:45.714232 12945 heartbeater.cc:464] Registering TS with master...
I20260924 00:21:45.714960 12945 heartbeater.cc:511] Master 127.10.4.125:46875 requested a full tablet report, sending...
I20260924 00:21:45.717343 12326 ts_manager.cc:194] Registered new tserver with Master: 08e4cf0195e645bcbb5ac5e3ede93096 (127.10.4.67:38139)
I20260924 00:21:45.729012 10257 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 00:21:45.776012 12404 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52892:
name: "default.Table_027aad11c8d7454bbf92402722920c2a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:45.778345 12403 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52892:
name: "default.Table_751162aa988a40a8a40c5f6c6c978860"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:45.779403 12402 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52892:
name: "default.Table_7e0e41deeaa74586ad39386be2a8244d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:45.780732 12401 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52892:
name: "default.Table_6888b4491c68473c8fbc8622820afc42"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:45.797401 12404 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_027aad11c8d7454bbf92402722920c2a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260924 00:21:45.799080 12401 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_6888b4491c68473c8fbc8622820afc42 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260924 00:21:45.801558 12403 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_751162aa988a40a8a40c5f6c6c978860 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260924 00:21:45.807690 12402 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_7e0e41deeaa74586ad39386be2a8244d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:45.822033 10257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu with pid 12362
I20260924 00:21:45.864626 12978 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dba2791dd9734eb4b84bbbf2f7d674b8)
I20260924 00:21:45.865237 12978 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:45.867969 12335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3c3a72a9440740a4948a000656ccabdf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fb563a88aa00432ba12db382b1435a39" is_pre_election: true
I20260924 00:21:45.867676 12978 leader_election.cc:288] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875), dba2791dd9734eb4b84bbbf2f7d674b8 (127.10.4.124:45407)
W20260924 00:21:45.873839 12239 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.124:45407: connect: Connection refused (error 111) [suppressed 12 similar messages]
W20260924 00:21:45.888127 12239 leader_election.cc:334] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer dba2791dd9734eb4b84bbbf2f7d674b8 (127.10.4.124:45407): Network error: Client connection negotiation failed: client connection to 127.10.4.124:45407: connect: Connection refused (error 111)
I20260924 00:21:45.888651 12239 leader_election.cc:302] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3a72a9440740a4948a000656ccabdf; no voters: dba2791dd9734eb4b84bbbf2f7d674b8, fb563a88aa00432ba12db382b1435a39
I20260924 00:21:45.889413 12978 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260924 00:21:45.917620 12800 heartbeater.cc:644] Failed to heartbeat to 127.10.4.124:45407 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.4.124:45407: connect: Connection refused (error 111)
W20260924 00:21:45.852128 12957 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.10.4.124:45407)
W20260924 00:21:46.002121 12957 outbound_call.cc:322] RPC callback for RPC call kudu.master.MasterService.CreateTable -> {remote=127.10.4.124:45407, user_credentials={real_user=slave}} blocked reactor thread for 151862us
I20260924 00:21:46.077720 12982 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dba2791dd9734eb4b84bbbf2f7d674b8)
I20260924 00:21:46.078612 12982 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:46.094676 12262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fb563a88aa00432ba12db382b1435a39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3a72a9440740a4948a000656ccabdf" is_pre_election: true
I20260924 00:21:46.095268 12262 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb563a88aa00432ba12db382b1435a39 in term 1.
I20260924 00:21:46.098084 12313 leader_election.cc:302] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39; no voters:
I20260924 00:21:46.100890 12982 leader_election.cc:288] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), dba2791dd9734eb4b84bbbf2f7d674b8 (127.10.4.124:45407)
I20260924 00:21:46.101486 12982 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 00:21:46.101774 12982 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [term 1 FOLLOWER]: Starting leader election (detected failure of leader dba2791dd9734eb4b84bbbf2f7d674b8)
I20260924 00:21:46.102165 12982 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [term 1 FOLLOWER]: Advancing to term 2
I20260924 00:21:46.118119 12982 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:46.120026 12982 leader_election.cc:288] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), dba2791dd9734eb4b84bbbf2f7d674b8 (127.10.4.124:45407)
I20260924 00:21:46.125905 12262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fb563a88aa00432ba12db382b1435a39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3a72a9440740a4948a000656ccabdf"
I20260924 00:21:46.126473 12262 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 1 FOLLOWER]: Advancing to term 2
W20260924 00:21:46.126853 12313 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.124:45407: connect: Connection refused (error 111) [suppressed 6 similar messages]
I20260924 00:21:46.137002 12262 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb563a88aa00432ba12db382b1435a39 in term 2.
I20260924 00:21:46.141541 12313 leader_election.cc:302] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39; no voters:
I20260924 00:21:46.142253 12982 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 00:21:46.142709 12982 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [term 2 LEADER]: Becoming Leader. State: Replica: fb563a88aa00432ba12db382b1435a39, State: Running, Role: LEADER
I20260924 00:21:46.145853 12982 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
W20260924 00:21:46.148466 12654 heartbeater.cc:644] Failed to heartbeat to 127.10.4.124:45407 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.4.124:45407: connect: Connection refused (error 111)
I20260924 00:21:46.150472 12983 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb563a88aa00432ba12db382b1435a39. Latest consensus state: current_term: 2 leader_uuid: "fb563a88aa00432ba12db382b1435a39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } } }
I20260924 00:21:46.151294 12983 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [sys.catalog]: This master's current role is: LEADER
W20260924 00:21:46.153244 12313 leader_election.cc:334] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer dba2791dd9734eb4b84bbbf2f7d674b8 (127.10.4.124:45407): Network error: Client connection negotiation failed: client connection to 127.10.4.124:45407: connect: Connection refused (error 111)
W20260924 00:21:46.153955 12313 leader_election.cc:334] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer dba2791dd9734eb4b84bbbf2f7d674b8 (127.10.4.124:45407): Network error: Client connection negotiation failed: client connection to 127.10.4.124:45407: connect: Connection refused (error 111)
I20260924 00:21:46.161804 12988 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:21:46.168859 12988 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:21:46.173617 12988 catalog_manager.cc:1259] Loaded cluster ID: 575a504d93ea472a8cb263817ca44ba4
I20260924 00:21:46.173831 12988 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:21:46.177403 12988 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:21:46.180812 12988 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39: Loaded TSK: 0
I20260924 00:21:46.182340 12988 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:21:46.185261 12988 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260924 00:21:46.189426 12988 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260924 00:21:46.207056 12944 heartbeater.cc:503] Master 127.10.4.124:45407 was elected leader, sending a full tablet report...
W20260924 00:21:46.209764 12944 heartbeater.cc:644] Failed to heartbeat to 127.10.4.124:45407 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.4.124:45407: connect: Connection refused (error 111)
I20260924 00:21:46.212987 12324 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33172:
name: "default.Table_751162aa988a40a8a40c5f6c6c978860"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:46.213922 12323 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33172:
name: "default.Table_027aad11c8d7454bbf92402722920c2a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:46.213505 12325 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33172:
name: "default.Table_7e0e41deeaa74586ad39386be2a8244d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:46.212599 12326 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33172:
name: "default.Table_6888b4491c68473c8fbc8622820afc42"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:21:46.253265 12324 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_751162aa988a40a8a40c5f6c6c978860 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260924 00:21:46.253279 12323 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_027aad11c8d7454bbf92402722920c2a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260924 00:21:46.261796 12326 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_6888b4491c68473c8fbc8622820afc42 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260924 00:21:46.261783 12325 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_7e0e41deeaa74586ad39386be2a8244d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:21:46.410652 10257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu
/tmp/dist-test-taskko7HPF/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.4.124:45407
--webserver_interface=127.10.4.124
--webserver_port=36597
--builtin_ntp_servers=127.10.4.84:44801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.4.126:37913,127.10.4.125:46875,127.10.4.124:45407
--hive_metastore_uris=thrift://127.0.0.1:45439
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260924 00:21:46.450434 12323 catalog_manager.cc:2148] created HMS catalog entry for table [id=291e8d50dc5248c280e4fde27a818707]
W20260924 00:21:46.474598 12313 consensus_peers.cc:487] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 -> Peer dba2791dd9734eb4b84bbbf2f7d674b8 (127.10.4.124:45407): Couldn't send request to peer dba2791dd9734eb4b84bbbf2f7d674b8. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.124:45407: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 00:21:46.477413 12262 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 2 FOLLOWER]: Refusing update from remote peer fb563a88aa00432ba12db382b1435a39: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20260924 00:21:46.479120 12983 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 00:21:46.516837 12992 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb563a88aa00432ba12db382b1435a39. Latest consensus state: current_term: 2 leader_uuid: "fb563a88aa00432ba12db382b1435a39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } } }
I20260924 00:21:46.517683 12992 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:21:46.519520 12991 mvcc.cc:205] Tried to move back new op lower bound from 7332697319292825600 to 7332697317982625792. Current Snapshot: MvccSnapshot[applied={T|T < 7332697319292825600}]
I20260924 00:21:46.523147 12983 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "fb563a88aa00432ba12db382b1435a39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } } }
I20260924 00:21:46.524144 12983 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39 [sys.catalog]: This master's current role is: LEADER
I20260924 00:21:46.535269 12992 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "fb563a88aa00432ba12db382b1435a39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } } }
I20260924 00:21:46.535931 12992 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:21:46.542647 12323 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33172:
table { table_name: "default.Table_027aad11c8d7454bbf92402722920c2a" } new_table_name: "default.Table_d5742bff56654aa0b9199a9b1bb928ca" modify_external_catalogs: true
I20260924 00:21:46.665441 12588 tablet_service.cc:1467] Processing CreateTablet for tablet 15ec845ede5b4ab786e5722d2b2888fb (DEFAULT_TABLE table=default.table_027aad11c8d7454bbf92402722920c2a [id=291e8d50dc5248c280e4fde27a818707]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:46.667520 12588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15ec845ede5b4ab786e5722d2b2888fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:46.689078 12735 tablet_service.cc:1467] Processing CreateTablet for tablet 15ec845ede5b4ab786e5722d2b2888fb (DEFAULT_TABLE table=default.table_027aad11c8d7454bbf92402722920c2a [id=291e8d50dc5248c280e4fde27a818707]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:46.690860 12735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15ec845ede5b4ab786e5722d2b2888fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:46.708268 12879 tablet_service.cc:1467] Processing CreateTablet for tablet 15ec845ede5b4ab786e5722d2b2888fb (DEFAULT_TABLE table=default.table_027aad11c8d7454bbf92402722920c2a [id=291e8d50dc5248c280e4fde27a818707]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:46.710415 12879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15ec845ede5b4ab786e5722d2b2888fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:46.722402 12945 heartbeater.cc:503] Master 127.10.4.125:46875 was elected leader, sending a full tablet report...
I20260924 00:21:46.742664 12322 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33172:
name: "default.Table_4a8f7937d4ef462592a1c80eefab1d0d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:46.751142 12321 master_service.cc:946] ReplaceTablet: received request to replace tablet 15ec845ede5b4ab786e5722d2b2888fb from {username='slave'} at 127.0.0.1:33172
I20260924 00:21:46.755982 12320 master_service.cc:946] ReplaceTablet: received request to replace tablet 15ec845ede5b4ab786e5722d2b2888fb from {username='slave'} at 127.0.0.1:33172
I20260924 00:21:46.761075 13004 tablet_bootstrap.cc:492] T 15ec845ede5b4ab786e5722d2b2888fb P 29842918acc045d29ea0fa4adfff5b9c: Bootstrap starting.
I20260924 00:21:46.771867 12321 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39: Sending DeleteTablet for 3 replicas of tablet 15ec845ede5b4ab786e5722d2b2888fb
I20260924 00:21:46.773488 12321 catalog_manager.cc:6565] ReplaceTablet: tablet 15ec845ede5b4ab786e5722d2b2888fb deleted and replaced by tablet adea85175645429fa62fe58ee5104f94
I20260924 00:21:46.774328 12878 tablet_service.cc:1514] Processing DeleteTablet for tablet 15ec845ede5b4ab786e5722d2b2888fb with delete_type TABLET_DATA_DELETED (replaced by tablet adea85175645429fa62fe58ee5104f94) from {username='slave'} at 127.0.0.1:36822
I20260924 00:21:46.779860 12734 tablet_service.cc:1514] Processing DeleteTablet for tablet 15ec845ede5b4ab786e5722d2b2888fb with delete_type TABLET_DATA_DELETED (replaced by tablet adea85175645429fa62fe58ee5104f94) from {username='slave'} at 127.0.0.1:38968
I20260924 00:21:46.782140 12588 tablet_service.cc:1514] Processing DeleteTablet for tablet 15ec845ede5b4ab786e5722d2b2888fb with delete_type TABLET_DATA_DELETED (replaced by tablet adea85175645429fa62fe58ee5104f94) from {username='slave'} at 127.0.0.1:33370
W20260924 00:21:46.791980 12313 catalog_manager.cc:4821] TS 08e4cf0195e645bcbb5ac5e3ede93096 (127.10.4.67:38139): delete failed for tablet 15ec845ede5b4ab786e5722d2b2888fb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 15ec845ede5b4ab786e5722d2b2888fb already in progress: creating tablet
I20260924 00:21:46.797617 12733 tablet_service.cc:1467] Processing CreateTablet for tablet adea85175645429fa62fe58ee5104f94 (DEFAULT_TABLE table=default.table_027aad11c8d7454bbf92402722920c2a [id=291e8d50dc5248c280e4fde27a818707]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:46.798851 12733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adea85175645429fa62fe58ee5104f94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:46.798923 13008 tablet_bootstrap.cc:492] T 15ec845ede5b4ab786e5722d2b2888fb P 08e4cf0195e645bcbb5ac5e3ede93096: Bootstrap starting.
I20260924 00:21:46.801957 12587 tablet_service.cc:1467] Processing CreateTablet for tablet adea85175645429fa62fe58ee5104f94 (DEFAULT_TABLE table=default.table_027aad11c8d7454bbf92402722920c2a [id=291e8d50dc5248c280e4fde27a818707]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:46.803038 12587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adea85175645429fa62fe58ee5104f94. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:21:46.804203 12310 catalog_manager.cc:4821] TS fa329f7568e1416597742422ff0026cb (127.10.4.66:40645): delete failed for tablet 15ec845ede5b4ab786e5722d2b2888fb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 15ec845ede5b4ab786e5722d2b2888fb already in progress: creating tablet
I20260924 00:21:46.805179 13009 tablet_bootstrap.cc:492] T 15ec845ede5b4ab786e5722d2b2888fb P fa329f7568e1416597742422ff0026cb: Bootstrap starting.
I20260924 00:21:46.811422 12879 tablet_service.cc:1467] Processing CreateTablet for tablet adea85175645429fa62fe58ee5104f94 (DEFAULT_TABLE table=default.table_027aad11c8d7454bbf92402722920c2a [id=291e8d50dc5248c280e4fde27a818707]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:46.813493 12879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adea85175645429fa62fe58ee5104f94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:46.823757 13009 tablet_bootstrap.cc:654] T 15ec845ede5b4ab786e5722d2b2888fb P fa329f7568e1416597742422ff0026cb: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:46.825345 13004 tablet_bootstrap.cc:654] T 15ec845ede5b4ab786e5722d2b2888fb P 29842918acc045d29ea0fa4adfff5b9c: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:46.826107 13009 log.cc:824] T 15ec845ede5b4ab786e5722d2b2888fb P fa329f7568e1416597742422ff0026cb: Log is configured to *not* fsync() on all Append() calls
I20260924 00:21:46.829536 13008 tablet_bootstrap.cc:654] T 15ec845ede5b4ab786e5722d2b2888fb P 08e4cf0195e645bcbb5ac5e3ede93096: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:46.831791 13008 log.cc:824] T 15ec845ede5b4ab786e5722d2b2888fb P 08e4cf0195e645bcbb5ac5e3ede93096: Log is configured to *not* fsync() on all Append() calls
I20260924 00:21:46.832655 13004 log.cc:824] T 15ec845ede5b4ab786e5722d2b2888fb P 29842918acc045d29ea0fa4adfff5b9c: Log is configured to *not* fsync() on all Append() calls
W20260924 00:21:46.834271 12313 catalog_manager.cc:4821] TS 29842918acc045d29ea0fa4adfff5b9c (127.10.4.65:46421): delete failed for tablet 15ec845ede5b4ab786e5722d2b2888fb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 15ec845ede5b4ab786e5722d2b2888fb already in progress: creating tablet
I20260924 00:21:46.845660 13009 tablet_bootstrap.cc:492] T 15ec845ede5b4ab786e5722d2b2888fb P fa329f7568e1416597742422ff0026cb: No bootstrap required, opened a new log
I20260924 00:21:46.846210 13009 ts_tablet_manager.cc:1397] T 15ec845ede5b4ab786e5722d2b2888fb P fa329f7568e1416597742422ff0026cb: Time spent bootstrapping tablet: real 0.042s user 0.014s sys 0.000s
I20260924 00:21:46.852499 13008 tablet_bootstrap.cc:492] T 15ec845ede5b4ab786e5722d2b2888fb P 08e4cf0195e645bcbb5ac5e3ede93096: No bootstrap required, opened a new log
I20260924 00:21:46.853012 13008 ts_tablet_manager.cc:1397] T 15ec845ede5b4ab786e5722d2b2888fb P 08e4cf0195e645bcbb5ac5e3ede93096: Time spent bootstrapping tablet: real 0.055s user 0.015s sys 0.000s
00:21:46.863 [HMS - ERROR - pool-6-thread-3] (RetryingHMSHandler.java:201) AlreadyExistsException(message:Table hive.default.table_751162aa988a40a8a40c5f6c6c978860 already exists)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1839)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1786)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(HiveMetaStore.java:2035)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
at com.sun.proxy.$Proxy25.create_table_with_environment_context(Unknown Source)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14800)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14784)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:104)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
I20260924 00:21:46.867980 13004 tablet_bootstrap.cc:492] T 15ec845ede5b4ab786e5722d2b2888fb P 29842918acc045d29ea0fa4adfff5b9c: No bootstrap required, opened a new log
I20260924 00:21:46.868510 13004 ts_tablet_manager.cc:1397] T 15ec845ede5b4ab786e5722d2b2888fb P 29842918acc045d29ea0fa4adfff5b9c: Time spent bootstrapping tablet: real 0.111s user 0.014s sys 0.009s
W20260924 00:21:46.875231 12324 catalog_manager.cc:2144] Already present: failed to create HMS catalog entry for table [id=d3580a8909b34618aa687663c37cc9e5]: failed to create Hive MetaStore table: TException - service has thrown: AlreadyExistsException(message=Table hive.default.table_751162aa988a40a8a40c5f6c6c978860 already exists)
I20260924 00:21:46.879978 12324 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33172:
name: "default.Table_e720de17aede4a629b383fe3a2b473fe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:46.902896 13004 raft_consensus.cc:348] T 15ec845ede5b4ab786e5722d2b2888fb P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } }
I20260924 00:21:46.903846 13004 raft_consensus.cc:374] T 15ec845ede5b4ab786e5722d2b2888fb P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:46.904160 13004 raft_consensus.cc:729] T 15ec845ede5b4ab786e5722d2b2888fb P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29842918acc045d29ea0fa4adfff5b9c, State: Initialized, Role: FOLLOWER
I20260924 00:21:46.905083 13004 consensus_queue.cc:260] T 15ec845ede5b4ab786e5722d2b2888fb P 29842918acc045d29ea0fa4adfff5b9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } }
I20260924 00:21:46.916209 12801 heartbeater.cc:503] Master 127.10.4.125:46875 was elected leader, sending a full tablet report...
I20260924 00:21:46.921270 12655 heartbeater.cc:503] Master 127.10.4.125:46875 was elected leader, sending a full tablet report...
I20260924 00:21:46.920790 13004 ts_tablet_manager.cc:1428] T 15ec845ede5b4ab786e5722d2b2888fb P 29842918acc045d29ea0fa4adfff5b9c: Time spent starting tablet: real 0.052s user 0.034s sys 0.011s
I20260924 00:21:46.933032 12587 tablet_service.cc:1514] Processing DeleteTablet for tablet 15ec845ede5b4ab786e5722d2b2888fb with delete_type TABLET_DATA_DELETED (replaced by tablet adea85175645429fa62fe58ee5104f94) from {username='slave'} at 127.0.0.1:33370
I20260924 00:21:46.933951 13007 tablet_replica.cc:330] T 15ec845ede5b4ab786e5722d2b2888fb P 29842918acc045d29ea0fa4adfff5b9c: stopping tablet replica
I20260924 00:21:46.934715 13007 raft_consensus.cc:2229] T 15ec845ede5b4ab786e5722d2b2888fb P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:46.935237 13007 raft_consensus.cc:2258] T 15ec845ede5b4ab786e5722d2b2888fb P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:46.936403 13004 tablet_bootstrap.cc:492] T adea85175645429fa62fe58ee5104f94 P 29842918acc045d29ea0fa4adfff5b9c: Bootstrap starting.
I20260924 00:21:46.937840 12733 tablet_service.cc:1514] Processing DeleteTablet for tablet 15ec845ede5b4ab786e5722d2b2888fb with delete_type TABLET_DATA_DELETED (replaced by tablet adea85175645429fa62fe58ee5104f94) from {username='slave'} at 127.0.0.1:38968
I20260924 00:21:46.938647 13009 raft_consensus.cc:348] T 15ec845ede5b4ab786e5722d2b2888fb P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } }
W20260924 00:21:46.939463 12310 catalog_manager.cc:4821] TS fa329f7568e1416597742422ff0026cb (127.10.4.66:40645): delete failed for tablet 15ec845ede5b4ab786e5722d2b2888fb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 15ec845ede5b4ab786e5722d2b2888fb already in progress: creating tablet
I20260924 00:21:46.939666 13009 raft_consensus.cc:374] T 15ec845ede5b4ab786e5722d2b2888fb P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:46.942319 13007 ts_tablet_manager.cc:1905] T 15ec845ede5b4ab786e5722d2b2888fb P 29842918acc045d29ea0fa4adfff5b9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:46.939946 13009 raft_consensus.cc:729] T 15ec845ede5b4ab786e5722d2b2888fb P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa329f7568e1416597742422ff0026cb, State: Initialized, Role: FOLLOWER
I20260924 00:21:46.943953 13004 tablet_bootstrap.cc:654] T adea85175645429fa62fe58ee5104f94 P 29842918acc045d29ea0fa4adfff5b9c: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:46.943785 13009 consensus_queue.cc:260] T 15ec845ede5b4ab786e5722d2b2888fb P fa329f7568e1416597742422ff0026cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } }
I20260924 00:21:46.946583 13008 raft_consensus.cc:348] T 15ec845ede5b4ab786e5722d2b2888fb P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } }
I20260924 00:21:46.947592 13008 raft_consensus.cc:374] T 15ec845ede5b4ab786e5722d2b2888fb P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:46.947906 13008 raft_consensus.cc:729] T 15ec845ede5b4ab786e5722d2b2888fb P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08e4cf0195e645bcbb5ac5e3ede93096, State: Initialized, Role: FOLLOWER
I20260924 00:21:46.948776 13008 consensus_queue.cc:260] T 15ec845ede5b4ab786e5722d2b2888fb P 08e4cf0195e645bcbb5ac5e3ede93096 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } }
I20260924 00:21:46.956529 13009 ts_tablet_manager.cc:1428] T 15ec845ede5b4ab786e5722d2b2888fb P fa329f7568e1416597742422ff0026cb: Time spent starting tablet: real 0.110s user 0.026s sys 0.011s
I20260924 00:21:46.959511 13009 tablet_bootstrap.cc:492] T adea85175645429fa62fe58ee5104f94 P fa329f7568e1416597742422ff0026cb: Bootstrap starting.
I20260924 00:21:46.963058 12733 tablet_service.cc:1514] Processing DeleteTablet for tablet 15ec845ede5b4ab786e5722d2b2888fb with delete_type TABLET_DATA_DELETED (replaced by tablet adea85175645429fa62fe58ee5104f94) from {username='slave'} at 127.0.0.1:38968
I20260924 00:21:46.963927 13006 tablet_replica.cc:330] T 15ec845ede5b4ab786e5722d2b2888fb P fa329f7568e1416597742422ff0026cb: stopping tablet replica
I20260924 00:21:46.964686 13006 raft_consensus.cc:2229] T 15ec845ede5b4ab786e5722d2b2888fb P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:46.965262 13006 raft_consensus.cc:2258] T 15ec845ede5b4ab786e5722d2b2888fb P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:46.964887 13009 tablet_bootstrap.cc:654] T adea85175645429fa62fe58ee5104f94 P fa329f7568e1416597742422ff0026cb: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:46.968504 13006 ts_tablet_manager.cc:1905] T 15ec845ede5b4ab786e5722d2b2888fb P fa329f7568e1416597742422ff0026cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:46.975531 13008 ts_tablet_manager.cc:1428] T 15ec845ede5b4ab786e5722d2b2888fb P 08e4cf0195e645bcbb5ac5e3ede93096: Time spent starting tablet: real 0.122s user 0.037s sys 0.004s
I20260924 00:21:46.980402 13007 ts_tablet_manager.cc:1918] T 15ec845ede5b4ab786e5722d2b2888fb P 29842918acc045d29ea0fa4adfff5b9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:46.980831 13007 log.cc:1196] T 15ec845ede5b4ab786e5722d2b2888fb P 29842918acc045d29ea0fa4adfff5b9c: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/15ec845ede5b4ab786e5722d2b2888fb
I20260924 00:21:46.983779 13007 ts_tablet_manager.cc:1939] T 15ec845ede5b4ab786e5722d2b2888fb P 29842918acc045d29ea0fa4adfff5b9c: Deleting consensus metadata
I20260924 00:21:46.985253 12321 master_service.cc:946] ReplaceTablet: received request to replace tablet 15ec845ede5b4ab786e5722d2b2888fb from {username='slave'} at 127.0.0.1:33172
I20260924 00:21:46.986626 12313 catalog_manager.cc:4841] TS 29842918acc045d29ea0fa4adfff5b9c (127.10.4.65:46421): tablet 15ec845ede5b4ab786e5722d2b2888fb (table default.table_027aad11c8d7454bbf92402722920c2a [id=291e8d50dc5248c280e4fde27a818707]) successfully deleted
I20260924 00:21:46.989387 12320 master_service.cc:946] ReplaceTablet: received request to replace tablet adea85175645429fa62fe58ee5104f94 from {username='slave'} at 127.0.0.1:33172
I20260924 00:21:47.003345 13004 tablet_bootstrap.cc:492] T adea85175645429fa62fe58ee5104f94 P 29842918acc045d29ea0fa4adfff5b9c: No bootstrap required, opened a new log
I20260924 00:21:47.003782 13004 ts_tablet_manager.cc:1397] T adea85175645429fa62fe58ee5104f94 P 29842918acc045d29ea0fa4adfff5b9c: Time spent bootstrapping tablet: real 0.068s user 0.010s sys 0.014s
I20260924 00:21:47.004087 13009 tablet_bootstrap.cc:492] T adea85175645429fa62fe58ee5104f94 P fa329f7568e1416597742422ff0026cb: No bootstrap required, opened a new log
I20260924 00:21:47.004493 13009 ts_tablet_manager.cc:1397] T adea85175645429fa62fe58ee5104f94 P fa329f7568e1416597742422ff0026cb: Time spent bootstrapping tablet: real 0.045s user 0.025s sys 0.010s
I20260924 00:21:47.006301 13004 raft_consensus.cc:348] T adea85175645429fa62fe58ee5104f94 P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:47.006979 13004 raft_consensus.cc:374] T adea85175645429fa62fe58ee5104f94 P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:47.006955 13009 raft_consensus.cc:348] T adea85175645429fa62fe58ee5104f94 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:47.007390 13004 raft_consensus.cc:729] T adea85175645429fa62fe58ee5104f94 P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29842918acc045d29ea0fa4adfff5b9c, State: Initialized, Role: FOLLOWER
I20260924 00:21:47.007623 13009 raft_consensus.cc:374] T adea85175645429fa62fe58ee5104f94 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:47.007879 13009 raft_consensus.cc:729] T adea85175645429fa62fe58ee5104f94 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa329f7568e1416597742422ff0026cb, State: Initialized, Role: FOLLOWER
I20260924 00:21:47.008154 13004 consensus_queue.cc:260] T adea85175645429fa62fe58ee5104f94 P 29842918acc045d29ea0fa4adfff5b9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:47.010360 13004 ts_tablet_manager.cc:1428] T adea85175645429fa62fe58ee5104f94 P 29842918acc045d29ea0fa4adfff5b9c: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260924 00:21:47.010519 13008 tablet_bootstrap.cc:492] T adea85175645429fa62fe58ee5104f94 P 08e4cf0195e645bcbb5ac5e3ede93096: Bootstrap starting.
I20260924 00:21:47.012530 13006 ts_tablet_manager.cc:1918] T 15ec845ede5b4ab786e5722d2b2888fb P fa329f7568e1416597742422ff0026cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:47.012871 13006 log.cc:1196] T 15ec845ede5b4ab786e5722d2b2888fb P fa329f7568e1416597742422ff0026cb: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/15ec845ede5b4ab786e5722d2b2888fb
I20260924 00:21:47.013732 13006 ts_tablet_manager.cc:1939] T 15ec845ede5b4ab786e5722d2b2888fb P fa329f7568e1416597742422ff0026cb: Deleting consensus metadata
I20260924 00:21:47.016304 12310 catalog_manager.cc:4841] TS fa329f7568e1416597742422ff0026cb (127.10.4.66:40645): tablet 15ec845ede5b4ab786e5722d2b2888fb (table default.table_027aad11c8d7454bbf92402722920c2a [id=291e8d50dc5248c280e4fde27a818707]) successfully deleted
I20260924 00:21:47.008473 13009 consensus_queue.cc:260] T adea85175645429fa62fe58ee5104f94 P fa329f7568e1416597742422ff0026cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:47.018471 13009 ts_tablet_manager.cc:1428] T adea85175645429fa62fe58ee5104f94 P fa329f7568e1416597742422ff0026cb: Time spent starting tablet: real 0.014s user 0.003s sys 0.003s
I20260924 00:21:47.031008 12320 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39: Sending DeleteTablet for 3 replicas of tablet adea85175645429fa62fe58ee5104f94
I20260924 00:21:47.033113 12587 tablet_service.cc:1514] Processing DeleteTablet for tablet adea85175645429fa62fe58ee5104f94 with delete_type TABLET_DATA_DELETED (replaced by tablet 6f3c0fe7b27d4177aaaba22f1ac04b27) from {username='slave'} at 127.0.0.1:33370
I20260924 00:21:47.033881 13007 tablet_replica.cc:330] T adea85175645429fa62fe58ee5104f94 P 29842918acc045d29ea0fa4adfff5b9c: stopping tablet replica
I20260924 00:21:47.034451 13007 raft_consensus.cc:2229] T adea85175645429fa62fe58ee5104f94 P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:47.034839 13007 raft_consensus.cc:2258] T adea85175645429fa62fe58ee5104f94 P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:47.036775 13007 ts_tablet_manager.cc:1905] T adea85175645429fa62fe58ee5104f94 P 29842918acc045d29ea0fa4adfff5b9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:47.037866 13008 tablet_bootstrap.cc:654] T adea85175645429fa62fe58ee5104f94 P 08e4cf0195e645bcbb5ac5e3ede93096: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:47.044317 12879 tablet_service.cc:1514] Processing DeleteTablet for tablet 15ec845ede5b4ab786e5722d2b2888fb with delete_type TABLET_DATA_DELETED (replaced by tablet adea85175645429fa62fe58ee5104f94) from {username='slave'} at 127.0.0.1:36822
I20260924 00:21:47.045183 13005 tablet_replica.cc:330] T 15ec845ede5b4ab786e5722d2b2888fb P 08e4cf0195e645bcbb5ac5e3ede93096: stopping tablet replica
I20260924 00:21:47.045719 13008 tablet_bootstrap.cc:492] T adea85175645429fa62fe58ee5104f94 P 08e4cf0195e645bcbb5ac5e3ede93096: No bootstrap required, opened a new log
I20260924 00:21:47.046007 13005 raft_consensus.cc:2229] T 15ec845ede5b4ab786e5722d2b2888fb P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:47.046114 13008 ts_tablet_manager.cc:1397] T adea85175645429fa62fe58ee5104f94 P 08e4cf0195e645bcbb5ac5e3ede93096: Time spent bootstrapping tablet: real 0.036s user 0.003s sys 0.007s
I20260924 00:21:47.046553 13005 raft_consensus.cc:2258] T 15ec845ede5b4ab786e5722d2b2888fb P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:47.049393 13005 ts_tablet_manager.cc:1905] T 15ec845ede5b4ab786e5722d2b2888fb P 08e4cf0195e645bcbb5ac5e3ede93096: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:47.049695 12733 tablet_service.cc:1514] Processing DeleteTablet for tablet adea85175645429fa62fe58ee5104f94 with delete_type TABLET_DATA_DELETED (replaced by tablet 6f3c0fe7b27d4177aaaba22f1ac04b27) from {username='slave'} at 127.0.0.1:38968
I20260924 00:21:47.054545 13006 tablet_replica.cc:330] T adea85175645429fa62fe58ee5104f94 P fa329f7568e1416597742422ff0026cb: stopping tablet replica
I20260924 00:21:47.055164 13006 raft_consensus.cc:2229] T adea85175645429fa62fe58ee5104f94 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:47.056351 13006 raft_consensus.cc:2258] T adea85175645429fa62fe58ee5104f94 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:47.058377 13006 ts_tablet_manager.cc:1905] T adea85175645429fa62fe58ee5104f94 P fa329f7568e1416597742422ff0026cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:47.058614 13008 raft_consensus.cc:348] T adea85175645429fa62fe58ee5104f94 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:47.059407 13008 raft_consensus.cc:374] T adea85175645429fa62fe58ee5104f94 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:47.059713 13008 raft_consensus.cc:729] T adea85175645429fa62fe58ee5104f94 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08e4cf0195e645bcbb5ac5e3ede93096, State: Initialized, Role: FOLLOWER
I20260924 00:21:47.062790 13005 ts_tablet_manager.cc:1918] T 15ec845ede5b4ab786e5722d2b2888fb P 08e4cf0195e645bcbb5ac5e3ede93096: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:47.063180 13005 log.cc:1196] T 15ec845ede5b4ab786e5722d2b2888fb P 08e4cf0195e645bcbb5ac5e3ede93096: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/15ec845ede5b4ab786e5722d2b2888fb
I20260924 00:21:47.064059 13005 ts_tablet_manager.cc:1939] T 15ec845ede5b4ab786e5722d2b2888fb P 08e4cf0195e645bcbb5ac5e3ede93096: Deleting consensus metadata
I20260924 00:21:47.064838 13008 consensus_queue.cc:260] T adea85175645429fa62fe58ee5104f94 P 08e4cf0195e645bcbb5ac5e3ede93096 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:47.066282 12733 tablet_service.cc:1514] Processing DeleteTablet for tablet adea85175645429fa62fe58ee5104f94 with delete_type TABLET_DATA_DELETED (replaced by tablet 6f3c0fe7b27d4177aaaba22f1ac04b27) from {username='slave'} at 127.0.0.1:38968
I20260924 00:21:47.067792 12313 catalog_manager.cc:4841] TS 08e4cf0195e645bcbb5ac5e3ede93096 (127.10.4.67:38139): tablet 15ec845ede5b4ab786e5722d2b2888fb (table default.table_027aad11c8d7454bbf92402722920c2a [id=291e8d50dc5248c280e4fde27a818707]) successfully deleted
W20260924 00:21:47.071177 12321 catalog_manager.cc:5328] Ignoring report from unknown tablet adea85175645429fa62fe58ee5104f94
I20260924 00:21:47.071520 13008 ts_tablet_manager.cc:1428] T adea85175645429fa62fe58ee5104f94 P 08e4cf0195e645bcbb5ac5e3ede93096: Time spent starting tablet: real 0.025s user 0.003s sys 0.003s
I20260924 00:21:47.075776 12320 catalog_manager.cc:6565] ReplaceTablet: tablet adea85175645429fa62fe58ee5104f94 deleted and replaced by tablet 6f3c0fe7b27d4177aaaba22f1ac04b27
I20260924 00:21:47.079308 12879 tablet_service.cc:1514] Processing DeleteTablet for tablet adea85175645429fa62fe58ee5104f94 with delete_type TABLET_DATA_DELETED (replaced by tablet 6f3c0fe7b27d4177aaaba22f1ac04b27) from {username='slave'} at 127.0.0.1:36822
I20260924 00:21:47.080236 13005 tablet_replica.cc:330] T adea85175645429fa62fe58ee5104f94 P 08e4cf0195e645bcbb5ac5e3ede93096: stopping tablet replica
I20260924 00:21:47.080864 13005 raft_consensus.cc:2229] T adea85175645429fa62fe58ee5104f94 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:47.081342 13005 raft_consensus.cc:2258] T adea85175645429fa62fe58ee5104f94 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:47.094458 13005 ts_tablet_manager.cc:1905] T adea85175645429fa62fe58ee5104f94 P 08e4cf0195e645bcbb5ac5e3ede93096: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:47.098188 13007 ts_tablet_manager.cc:1918] T adea85175645429fa62fe58ee5104f94 P 29842918acc045d29ea0fa4adfff5b9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:47.098714 13007 log.cc:1196] T adea85175645429fa62fe58ee5104f94 P 29842918acc045d29ea0fa4adfff5b9c: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/adea85175645429fa62fe58ee5104f94
I20260924 00:21:47.099437 13007 ts_tablet_manager.cc:1939] T adea85175645429fa62fe58ee5104f94 P 29842918acc045d29ea0fa4adfff5b9c: Deleting consensus metadata
W20260924 00:21:47.105809 12320 catalog_manager.cc:5328] Ignoring report from unknown tablet adea85175645429fa62fe58ee5104f94
I20260924 00:21:47.109365 13005 ts_tablet_manager.cc:1918] T adea85175645429fa62fe58ee5104f94 P 08e4cf0195e645bcbb5ac5e3ede93096: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:47.109944 13005 log.cc:1196] T adea85175645429fa62fe58ee5104f94 P 08e4cf0195e645bcbb5ac5e3ede93096: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/adea85175645429fa62fe58ee5104f94
I20260924 00:21:47.110574 13005 ts_tablet_manager.cc:1939] T adea85175645429fa62fe58ee5104f94 P 08e4cf0195e645bcbb5ac5e3ede93096: Deleting consensus metadata
I20260924 00:21:47.114291 12325 catalog_manager.cc:2148] created HMS catalog entry for table [id=0cf5ae7d6b56461fb17c887ec515c931]
I20260924 00:21:47.121995 13006 ts_tablet_manager.cc:1918] T adea85175645429fa62fe58ee5104f94 P fa329f7568e1416597742422ff0026cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:47.126140 12313 catalog_manager.cc:4841] TS 08e4cf0195e645bcbb5ac5e3ede93096 (127.10.4.67:38139): tablet adea85175645429fa62fe58ee5104f94 (table default.table_027aad11c8d7454bbf92402722920c2a [id=291e8d50dc5248c280e4fde27a818707]) successfully deleted
I20260924 00:21:47.126782 12313 catalog_manager.cc:4841] TS 29842918acc045d29ea0fa4adfff5b9c (127.10.4.65:46421): tablet adea85175645429fa62fe58ee5104f94 (table default.table_027aad11c8d7454bbf92402722920c2a [id=291e8d50dc5248c280e4fde27a818707]) successfully deleted
I20260924 00:21:47.135017 12587 tablet_service.cc:1467] Processing CreateTablet for tablet 6f3c0fe7b27d4177aaaba22f1ac04b27 (DEFAULT_TABLE table=default.table_027aad11c8d7454bbf92402722920c2a [id=291e8d50dc5248c280e4fde27a818707]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:47.135380 12879 tablet_service.cc:1467] Processing CreateTablet for tablet 6f3c0fe7b27d4177aaaba22f1ac04b27 (DEFAULT_TABLE table=default.table_027aad11c8d7454bbf92402722920c2a [id=291e8d50dc5248c280e4fde27a818707]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:47.136096 12587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f3c0fe7b27d4177aaaba22f1ac04b27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:47.136493 12879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f3c0fe7b27d4177aaaba22f1ac04b27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:47.139614 12733 tablet_service.cc:1467] Processing CreateTablet for tablet 6f3c0fe7b27d4177aaaba22f1ac04b27 (DEFAULT_TABLE table=default.table_027aad11c8d7454bbf92402722920c2a [id=291e8d50dc5248c280e4fde27a818707]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:47.140664 12733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f3c0fe7b27d4177aaaba22f1ac04b27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:47.151096 13008 tablet_bootstrap.cc:492] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P 08e4cf0195e645bcbb5ac5e3ede93096: Bootstrap starting.
I20260924 00:21:47.159034 13004 tablet_bootstrap.cc:492] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P 29842918acc045d29ea0fa4adfff5b9c: Bootstrap starting.
I20260924 00:21:47.160112 12325 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33172:
table { table_name: "default.Table_7e0e41deeaa74586ad39386be2a8244d" } new_table_name: "default.Table_f0b527f3d42148d69722944b46b07d72" modify_external_catalogs: true
I20260924 00:21:47.157203 13008 tablet_bootstrap.cc:654] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P 08e4cf0195e645bcbb5ac5e3ede93096: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:47.164492 13004 tablet_bootstrap.cc:654] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P 29842918acc045d29ea0fa4adfff5b9c: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:47.171046 13008 tablet_bootstrap.cc:492] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P 08e4cf0195e645bcbb5ac5e3ede93096: No bootstrap required, opened a new log
I20260924 00:21:47.171487 13008 ts_tablet_manager.cc:1397] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P 08e4cf0195e645bcbb5ac5e3ede93096: Time spent bootstrapping tablet: real 0.021s user 0.007s sys 0.004s
I20260924 00:21:47.174309 13008 raft_consensus.cc:348] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:47.175035 13008 raft_consensus.cc:374] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:47.175290 13008 raft_consensus.cc:729] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08e4cf0195e645bcbb5ac5e3ede93096, State: Initialized, Role: FOLLOWER
I20260924 00:21:47.175891 13008 consensus_queue.cc:260] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P 08e4cf0195e645bcbb5ac5e3ede93096 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:47.122493 13006 log.cc:1196] T adea85175645429fa62fe58ee5104f94 P fa329f7568e1416597742422ff0026cb: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/adea85175645429fa62fe58ee5104f94
I20260924 00:21:47.177711 13006 ts_tablet_manager.cc:1939] T adea85175645429fa62fe58ee5104f94 P fa329f7568e1416597742422ff0026cb: Deleting consensus metadata
I20260924 00:21:47.177736 13008 ts_tablet_manager.cc:1428] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P 08e4cf0195e645bcbb5ac5e3ede93096: Time spent starting tablet: real 0.006s user 0.004s sys 0.003s
I20260924 00:21:47.182240 12310 catalog_manager.cc:4841] TS fa329f7568e1416597742422ff0026cb (127.10.4.66:40645): tablet adea85175645429fa62fe58ee5104f94 (table default.table_027aad11c8d7454bbf92402722920c2a [id=291e8d50dc5248c280e4fde27a818707]) successfully deleted
I20260924 00:21:47.188382 13004 tablet_bootstrap.cc:492] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P 29842918acc045d29ea0fa4adfff5b9c: No bootstrap required, opened a new log
I20260924 00:21:47.193256 12735 tablet_service.cc:1467] Processing CreateTablet for tablet 1c3ffccda42840d88b039e39b9ecc2c7 (DEFAULT_TABLE table=default.table_7e0e41deeaa74586ad39386be2a8244d [id=0cf5ae7d6b56461fb17c887ec515c931]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:47.188804 13004 ts_tablet_manager.cc:1397] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P 29842918acc045d29ea0fa4adfff5b9c: Time spent bootstrapping tablet: real 0.030s user 0.010s sys 0.001s
I20260924 00:21:47.194386 12735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c3ffccda42840d88b039e39b9ecc2c7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 00:21:47.194332 12310 catalog_manager.cc:4808] TS fa329f7568e1416597742422ff0026cb (127.10.4.66:40645): delete failed for tablet adea85175645429fa62fe58ee5104f94 because the tablet was not found. No further retry: Not found: Tablet not found: adea85175645429fa62fe58ee5104f94
I20260924 00:21:47.196925 13004 raft_consensus.cc:348] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:47.197624 13004 raft_consensus.cc:374] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:47.197880 13004 raft_consensus.cc:729] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29842918acc045d29ea0fa4adfff5b9c, State: Initialized, Role: FOLLOWER
W20260924 00:21:47.201398 12951 tablet.cc:2369] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P 08e4cf0195e645bcbb5ac5e3ede93096: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:21:47.201792 13009 tablet_bootstrap.cc:492] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P fa329f7568e1416597742422ff0026cb: Bootstrap starting.
I20260924 00:21:47.201751 12879 tablet_service.cc:1467] Processing CreateTablet for tablet 1c3ffccda42840d88b039e39b9ecc2c7 (DEFAULT_TABLE table=default.table_7e0e41deeaa74586ad39386be2a8244d [id=0cf5ae7d6b56461fb17c887ec515c931]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:47.202744 12879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c3ffccda42840d88b039e39b9ecc2c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:47.201367 12587 tablet_service.cc:1467] Processing CreateTablet for tablet 1c3ffccda42840d88b039e39b9ecc2c7 (DEFAULT_TABLE table=default.table_7e0e41deeaa74586ad39386be2a8244d [id=0cf5ae7d6b56461fb17c887ec515c931]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:47.208164 12587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c3ffccda42840d88b039e39b9ecc2c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:47.209955 12320 master_service.cc:946] ReplaceTablet: received request to replace tablet 6f3c0fe7b27d4177aaaba22f1ac04b27 from {username='slave'} at 127.0.0.1:33172
I20260924 00:21:47.213786 13009 tablet_bootstrap.cc:654] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P fa329f7568e1416597742422ff0026cb: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:47.225721 13004 consensus_queue.cc:260] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P 29842918acc045d29ea0fa4adfff5b9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:47.231186 13009 tablet_bootstrap.cc:492] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P fa329f7568e1416597742422ff0026cb: No bootstrap required, opened a new log
I20260924 00:21:47.231657 13009 ts_tablet_manager.cc:1397] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P fa329f7568e1416597742422ff0026cb: Time spent bootstrapping tablet: real 0.030s user 0.007s sys 0.003s
I20260924 00:21:47.233011 12326 catalog_manager.cc:2148] created HMS catalog entry for table [id=670fab2a44eb41ee89d1a16ef0aef90c]
I20260924 00:21:47.234961 13009 raft_consensus.cc:348] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:47.235641 13009 raft_consensus.cc:374] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:47.235898 13009 raft_consensus.cc:729] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa329f7568e1416597742422ff0026cb, State: Initialized, Role: FOLLOWER
I20260924 00:21:47.236478 13009 consensus_queue.cc:260] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P fa329f7568e1416597742422ff0026cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:47.237128 13004 ts_tablet_manager.cc:1428] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P 29842918acc045d29ea0fa4adfff5b9c: Time spent starting tablet: real 0.043s user 0.006s sys 0.000s
I20260924 00:21:47.238303 13009 ts_tablet_manager.cc:1428] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P fa329f7568e1416597742422ff0026cb: Time spent starting tablet: real 0.006s user 0.001s sys 0.003s
F20260924 00:21:47.244529 12513 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_751162aa988a40a8a40c5f6c6c978860
*** Check failure stack trace: ***
I20260924 00:21:47.253033 13004 tablet_bootstrap.cc:492] T 1c3ffccda42840d88b039e39b9ecc2c7 P 29842918acc045d29ea0fa4adfff5b9c: Bootstrap starting.
I20260924 00:21:47.254091 12320 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39: Sending DeleteTablet for 3 replicas of tablet 6f3c0fe7b27d4177aaaba22f1ac04b27
I20260924 00:21:47.257082 12733 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f3c0fe7b27d4177aaaba22f1ac04b27 with delete_type TABLET_DATA_DELETED (replaced by tablet 228068c1c54e4e929e3153a366a30099) from {username='slave'} at 127.0.0.1:38968
I20260924 00:21:47.257777 13006 tablet_replica.cc:330] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P fa329f7568e1416597742422ff0026cb: stopping tablet replica
I20260924 00:21:47.258404 13006 raft_consensus.cc:2229] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:47.258823 13006 raft_consensus.cc:2258] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:47.261152 13006 ts_tablet_manager.cc:1905] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P fa329f7568e1416597742422ff0026cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:47.263070 13004 tablet_bootstrap.cc:654] T 1c3ffccda42840d88b039e39b9ecc2c7 P 29842918acc045d29ea0fa4adfff5b9c: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:47.264376 12587 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f3c0fe7b27d4177aaaba22f1ac04b27 with delete_type TABLET_DATA_DELETED (replaced by tablet 228068c1c54e4e929e3153a366a30099) from {username='slave'} at 127.0.0.1:33370
I20260924 00:21:47.265097 13007 tablet_replica.cc:330] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P 29842918acc045d29ea0fa4adfff5b9c: stopping tablet replica
I20260924 00:21:47.265776 13007 raft_consensus.cc:2229] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:47.266220 13007 raft_consensus.cc:2258] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:47.268437 13007 ts_tablet_manager.cc:1905] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P 29842918acc045d29ea0fa4adfff5b9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:47.279697 12326 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33172 (request call id 12) took 1068 ms. Trace:
I20260924 00:21:47.284686 12733 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f3c0fe7b27d4177aaaba22f1ac04b27 with delete_type TABLET_DATA_DELETED (replaced by tablet 228068c1c54e4e929e3153a366a30099) from {username='slave'} at 127.0.0.1:38968
I20260924 00:21:47.283591 12326 rpcz_store.cc:276] 0924 00:21:46.211348 (+ 0us) service_pool.cc:170] Inserting onto call queue
0924 00:21:46.211796 (+ 448us) service_pool.cc:227] Handling call
0924 00:21:46.263061 (+ 51265us) catalog_manager.cc:2075] Acquired catalog manager lock
0924 00:21:46.264698 (+ 1637us) catalog_manager.cc:2121] Created new table and tablet info
0924 00:21:47.232967 (+968269us) catalog_manager.cc:2147] Created new table in HMS catalog
0924 00:21:47.278994 (+ 46027us) catalog_manager.cc:2181] Wrote table and tablets to system table
0924 00:21:47.279335 (+ 341us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0924 00:21:47.279612 (+ 277us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 00:21:47.234662 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 00:21:47.235080 (+ 418us) write_op.cc:432] Acquiring schema lock in shared mode
0924 00:21:47.235107 (+ 27us) write_op.cc:435] Acquired schema lock
0924 00:21:47.235118 (+ 11us) tablet.cc:660] Decoding operations
0924 00:21:47.235412 (+ 294us) write_op.cc:620] Acquiring the partition lock for write op
0924 00:21:47.235453 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0924 00:21:47.235477 (+ 24us) tablet.cc:683] Acquiring locks for 2 operations
0924 00:21:47.235804 (+ 327us) tablet.cc:699] Row locks acquired
0924 00:21:47.235817 (+ 13us) write_op.cc:260] PREPARE: finished
0924 00:21:47.236117 (+ 300us) write_op.cc:270] Start()
0924 00:21:47.236261 (+ 144us) write_op.cc:276] Timestamp: P: 1790209307236053 usec, L: 0
0924 00:21:47.236276 (+ 15us) op_driver.cc:331] REPLICATION: starting
0924 00:21:47.236751 (+ 475us) log.cc:842] Serialized 432 byte log entry
0924 00:21:47.277160 (+ 40409us) op_driver.cc:444] REPLICATION: finished
0924 00:21:47.277566 (+ 406us) write_op.cc:301] APPLY: starting
0924 00:21:47.277621 (+ 55us) tablet.cc:1365] starting BulkCheckPresence
0924 00:21:47.277723 (+ 102us) tablet.cc:1368] finished BulkCheckPresence
0924 00:21:47.277813 (+ 90us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 00:21:47.278157 (+ 344us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 00:21:47.278296 (+ 139us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0924 00:21:47.278338 (+ 42us) write_op.cc:312] APPLY: finished
0924 00:21:47.278522 (+ 184us) log.cc:842] Serialized 28 byte log entry
0924 00:21:47.278665 (+ 143us) write_op.cc:489] Releasing partition, row and schema locks
0924 00:21:47.278734 (+ 69us) write_op.cc:454] Released schema lock
0924 00:21:47.278766 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0924 00:21:47.280136 (+ 1370us) write_op.cc:489] Releasing partition, row and schema locks
0924 00:21:47.280164 (+ 28us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":846306,"Hive Metastore.run_cpu_time_us":577,"Hive Metastore.run_wall_time_us":118572,"spinlock_wait_cycles":7040,"child_traces":[["op",{"apply.queue_time_us":260,"apply.run_cpu_time_us":1599,"apply.run_wall_time_us":2758,"num_ops":2,"prepare.queue_time_us":167,"prepare.run_cpu_time_us":2065,"prepare.run_wall_time_us":2382,"raft.queue_time_us":399,"raft.run_cpu_time_us":12,"raft.run_wall_time_us":13,"replication_time_us":40806}]]}
I20260924 00:21:47.297730 12320 catalog_manager.cc:6565] ReplaceTablet: tablet 6f3c0fe7b27d4177aaaba22f1ac04b27 deleted and replaced by tablet 228068c1c54e4e929e3153a366a30099
I20260924 00:21:47.300932 12587 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f3c0fe7b27d4177aaaba22f1ac04b27 with delete_type TABLET_DATA_DELETED (replaced by tablet 228068c1c54e4e929e3153a366a30099) from {username='slave'} at 127.0.0.1:33370
I20260924 00:21:47.303294 12320 master_service.cc:946] ReplaceTablet: received request to replace tablet 6f3c0fe7b27d4177aaaba22f1ac04b27 from {username='slave'} at 127.0.0.1:33172
I20260924 00:21:47.303743 12878 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f3c0fe7b27d4177aaaba22f1ac04b27 with delete_type TABLET_DATA_DELETED (replaced by tablet 228068c1c54e4e929e3153a366a30099) from {username='slave'} at 127.0.0.1:36822
I20260924 00:21:47.304502 13005 tablet_replica.cc:330] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P 08e4cf0195e645bcbb5ac5e3ede93096: stopping tablet replica
I20260924 00:21:47.305145 13005 raft_consensus.cc:2229] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:47.305572 13005 raft_consensus.cc:2258] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:47.307543 13005 ts_tablet_manager.cc:1905] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P 08e4cf0195e645bcbb5ac5e3ede93096: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:47.309203 12321 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33172:
table { table_name: "default.Table_6888b4491c68473c8fbc8622820afc42" } modify_external_catalogs: true
I20260924 00:21:47.310012 12321 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33172:
table { table_name: "default.Table_6888b4491c68473c8fbc8622820afc42" } modify_external_catalogs: true
@ 0x7f8876ffd9a0 google::LogMessage::Flush() at ??:0
I20260924 00:21:47.324345 12733 tablet_service.cc:1467] Processing CreateTablet for tablet 228068c1c54e4e929e3153a366a30099 (DEFAULT_TABLE table=default.table_027aad11c8d7454bbf92402722920c2a [id=291e8d50dc5248c280e4fde27a818707]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:47.325466 12733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 228068c1c54e4e929e3153a366a30099. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:47.327637 13007 ts_tablet_manager.cc:1918] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P 29842918acc045d29ea0fa4adfff5b9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:47.328034 13007 log.cc:1196] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P 29842918acc045d29ea0fa4adfff5b9c: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/6f3c0fe7b27d4177aaaba22f1ac04b27
I20260924 00:21:47.328703 13007 ts_tablet_manager.cc:1939] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P 29842918acc045d29ea0fa4adfff5b9c: Deleting consensus metadata
I20260924 00:21:47.331630 13009 tablet_bootstrap.cc:492] T 1c3ffccda42840d88b039e39b9ecc2c7 P fa329f7568e1416597742422ff0026cb: Bootstrap starting.
I20260924 00:21:47.337184 13009 tablet_bootstrap.cc:654] T 1c3ffccda42840d88b039e39b9ecc2c7 P fa329f7568e1416597742422ff0026cb: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:47.339476 12587 tablet_service.cc:1467] Processing CreateTablet for tablet 228068c1c54e4e929e3153a366a30099 (DEFAULT_TABLE table=default.table_027aad11c8d7454bbf92402722920c2a [id=291e8d50dc5248c280e4fde27a818707]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:47.340669 12587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 228068c1c54e4e929e3153a366a30099. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:47.346212 13008 tablet_bootstrap.cc:492] T 1c3ffccda42840d88b039e39b9ecc2c7 P 08e4cf0195e645bcbb5ac5e3ede93096: Bootstrap starting.
I20260924 00:21:47.351709 13008 tablet_bootstrap.cc:654] T 1c3ffccda42840d88b039e39b9ecc2c7 P 08e4cf0195e645bcbb5ac5e3ede93096: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:47.353223 12878 tablet_service.cc:1467] Processing CreateTablet for tablet 228068c1c54e4e929e3153a366a30099 (DEFAULT_TABLE table=default.table_027aad11c8d7454bbf92402722920c2a [id=291e8d50dc5248c280e4fde27a818707]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:47.354274 12878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 228068c1c54e4e929e3153a366a30099. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:47.356603 13006 ts_tablet_manager.cc:1918] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P fa329f7568e1416597742422ff0026cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:47.356999 13006 log.cc:1196] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P fa329f7568e1416597742422ff0026cb: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/6f3c0fe7b27d4177aaaba22f1ac04b27
I20260924 00:21:47.357606 13006 ts_tablet_manager.cc:1939] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P fa329f7568e1416597742422ff0026cb: Deleting consensus metadata
I20260924 00:21:47.362234 12310 catalog_manager.cc:4841] TS fa329f7568e1416597742422ff0026cb (127.10.4.66:40645): tablet 6f3c0fe7b27d4177aaaba22f1ac04b27 (table default.table_027aad11c8d7454bbf92402722920c2a [id=291e8d50dc5248c280e4fde27a818707]) successfully deleted
I20260924 00:21:47.364260 12313 catalog_manager.cc:4841] TS 29842918acc045d29ea0fa4adfff5b9c (127.10.4.65:46421): tablet 6f3c0fe7b27d4177aaaba22f1ac04b27 (table default.table_027aad11c8d7454bbf92402722920c2a [id=291e8d50dc5248c280e4fde27a818707]) successfully deleted
W20260924 00:21:47.365057 12313 catalog_manager.cc:4808] TS 29842918acc045d29ea0fa4adfff5b9c (127.10.4.65:46421): delete failed for tablet 6f3c0fe7b27d4177aaaba22f1ac04b27 because the tablet was not found. No further retry: Not found: Tablet not found: 6f3c0fe7b27d4177aaaba22f1ac04b27
I20260924 00:21:47.365486 12320 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33172:
name: "default.Table_b34b7e9f7270468b93a28c7329bbec21"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:47.372361 13005 ts_tablet_manager.cc:1918] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P 08e4cf0195e645bcbb5ac5e3ede93096: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
@ 0x7f8876ffd70f google::LogMessage::~LogMessage() at ??:0
I20260924 00:21:47.376916 13009 tablet_bootstrap.cc:492] T 1c3ffccda42840d88b039e39b9ecc2c7 P fa329f7568e1416597742422ff0026cb: No bootstrap required, opened a new log
I20260924 00:21:47.377447 13009 ts_tablet_manager.cc:1397] T 1c3ffccda42840d88b039e39b9ecc2c7 P fa329f7568e1416597742422ff0026cb: Time spent bootstrapping tablet: real 0.046s user 0.008s sys 0.004s
W20260924 00:21:47.382077 12310 catalog_manager.cc:4808] TS fa329f7568e1416597742422ff0026cb (127.10.4.66:40645): delete failed for tablet 6f3c0fe7b27d4177aaaba22f1ac04b27 because the tablet was not found. No further retry: Not found: Tablet not found: 6f3c0fe7b27d4177aaaba22f1ac04b27
I20260924 00:21:47.372772 13005 log.cc:1196] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P 08e4cf0195e645bcbb5ac5e3ede93096: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/6f3c0fe7b27d4177aaaba22f1ac04b27
I20260924 00:21:47.386065 13008 tablet_bootstrap.cc:492] T 1c3ffccda42840d88b039e39b9ecc2c7 P 08e4cf0195e645bcbb5ac5e3ede93096: No bootstrap required, opened a new log
I20260924 00:21:47.386472 13008 ts_tablet_manager.cc:1397] T 1c3ffccda42840d88b039e39b9ecc2c7 P 08e4cf0195e645bcbb5ac5e3ede93096: Time spent bootstrapping tablet: real 0.040s user 0.004s sys 0.007s
I20260924 00:21:47.388795 13008 raft_consensus.cc:348] T 1c3ffccda42840d88b039e39b9ecc2c7 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } }
I20260924 00:21:47.389490 13008 raft_consensus.cc:374] T 1c3ffccda42840d88b039e39b9ecc2c7 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:47.389740 13008 raft_consensus.cc:729] T 1c3ffccda42840d88b039e39b9ecc2c7 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08e4cf0195e645bcbb5ac5e3ede93096, State: Initialized, Role: FOLLOWER
I20260924 00:21:47.390682 13005 ts_tablet_manager.cc:1939] T 6f3c0fe7b27d4177aaaba22f1ac04b27 P 08e4cf0195e645bcbb5ac5e3ede93096: Deleting consensus metadata
I20260924 00:21:47.390841 13004 tablet_bootstrap.cc:492] T 1c3ffccda42840d88b039e39b9ecc2c7 P 29842918acc045d29ea0fa4adfff5b9c: No bootstrap required, opened a new log
I20260924 00:21:47.391403 13004 ts_tablet_manager.cc:1397] T 1c3ffccda42840d88b039e39b9ecc2c7 P 29842918acc045d29ea0fa4adfff5b9c: Time spent bootstrapping tablet: real 0.139s user 0.027s sys 0.007s
I20260924 00:21:47.393829 13004 raft_consensus.cc:348] T 1c3ffccda42840d88b039e39b9ecc2c7 P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } }
I20260924 00:21:47.394493 13004 raft_consensus.cc:374] T 1c3ffccda42840d88b039e39b9ecc2c7 P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:47.394737 13004 raft_consensus.cc:729] T 1c3ffccda42840d88b039e39b9ecc2c7 P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29842918acc045d29ea0fa4adfff5b9c, State: Initialized, Role: FOLLOWER
I20260924 00:21:47.395326 13004 consensus_queue.cc:260] T 1c3ffccda42840d88b039e39b9ecc2c7 P 29842918acc045d29ea0fa4adfff5b9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } }
I20260924 00:21:47.397511 13004 ts_tablet_manager.cc:1428] T 1c3ffccda42840d88b039e39b9ecc2c7 P 29842918acc045d29ea0fa4adfff5b9c: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260924 00:21:47.379724 13009 raft_consensus.cc:348] T 1c3ffccda42840d88b039e39b9ecc2c7 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } }
I20260924 00:21:47.398048 13009 raft_consensus.cc:374] T 1c3ffccda42840d88b039e39b9ecc2c7 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:47.398319 13009 raft_consensus.cc:729] T 1c3ffccda42840d88b039e39b9ecc2c7 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa329f7568e1416597742422ff0026cb, State: Initialized, Role: FOLLOWER
I20260924 00:21:47.401038 13008 consensus_queue.cc:260] T 1c3ffccda42840d88b039e39b9ecc2c7 P 08e4cf0195e645bcbb5ac5e3ede93096 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } }
I20260924 00:21:47.402140 13009 consensus_queue.cc:260] T 1c3ffccda42840d88b039e39b9ecc2c7 P fa329f7568e1416597742422ff0026cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } }
@ 0x7f8882098aea kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260924 00:21:47.411393 13008 ts_tablet_manager.cc:1428] T 1c3ffccda42840d88b039e39b9ecc2c7 P 08e4cf0195e645bcbb5ac5e3ede93096: Time spent starting tablet: real 0.025s user 0.006s sys 0.000s
I20260924 00:21:47.418376 12313 catalog_manager.cc:4841] TS 08e4cf0195e645bcbb5ac5e3ede93096 (127.10.4.67:38139): tablet 6f3c0fe7b27d4177aaaba22f1ac04b27 (table default.table_027aad11c8d7454bbf92402722920c2a [id=291e8d50dc5248c280e4fde27a818707]) successfully deleted
I20260924 00:21:47.420171 13009 ts_tablet_manager.cc:1428] T 1c3ffccda42840d88b039e39b9ecc2c7 P fa329f7568e1416597742422ff0026cb: Time spent starting tablet: real 0.042s user 0.004s sys 0.001s
W20260924 00:21:47.432078 12990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
@ 0x7f888209833c kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
W20260924 00:21:47.432840 12990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:21:47.433553 12990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260924 00:21:47.435074 12735 tablet_service.cc:1467] Processing CreateTablet for tablet 198628c8cdac4c0dab683eeb55ae4a87 (DEFAULT_TABLE table=default.table_6888b4491c68473c8fbc8622820afc42 [id=670fab2a44eb41ee89d1a16ef0aef90c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:47.436101 12735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 198628c8cdac4c0dab683eeb55ae4a87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:47.446627 12879 tablet_service.cc:1467] Processing CreateTablet for tablet 198628c8cdac4c0dab683eeb55ae4a87 (DEFAULT_TABLE table=default.table_6888b4491c68473c8fbc8622820afc42 [id=670fab2a44eb41ee89d1a16ef0aef90c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:47.446707 12588 tablet_service.cc:1467] Processing CreateTablet for tablet 198628c8cdac4c0dab683eeb55ae4a87 (DEFAULT_TABLE table=default.table_6888b4491c68473c8fbc8622820afc42 [id=670fab2a44eb41ee89d1a16ef0aef90c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:47.447656 12879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 198628c8cdac4c0dab683eeb55ae4a87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:47.447731 12588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 198628c8cdac4c0dab683eeb55ae4a87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:47.453315 13009 tablet_bootstrap.cc:492] T 228068c1c54e4e929e3153a366a30099 P fa329f7568e1416597742422ff0026cb: Bootstrap starting.
I20260924 00:21:47.462117 13004 tablet_bootstrap.cc:492] T 228068c1c54e4e929e3153a366a30099 P 29842918acc045d29ea0fa4adfff5b9c: Bootstrap starting.
I20260924 00:21:47.466068 13008 tablet_bootstrap.cc:492] T 228068c1c54e4e929e3153a366a30099 P 08e4cf0195e645bcbb5ac5e3ede93096: Bootstrap starting.
I20260924 00:21:47.467687 13004 tablet_bootstrap.cc:654] T 228068c1c54e4e929e3153a366a30099 P 29842918acc045d29ea0fa4adfff5b9c: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:47.468113 13009 tablet_bootstrap.cc:654] T 228068c1c54e4e929e3153a366a30099 P fa329f7568e1416597742422ff0026cb: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:47.471341 13008 tablet_bootstrap.cc:654] T 228068c1c54e4e929e3153a366a30099 P 08e4cf0195e645bcbb5ac5e3ede93096: Neither blocks nor log segments found. Creating new log.
@ 0x7f888209bfd2 kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260924 00:21:47.487893 13004 tablet_bootstrap.cc:492] T 228068c1c54e4e929e3153a366a30099 P 29842918acc045d29ea0fa4adfff5b9c: No bootstrap required, opened a new log
I20260924 00:21:47.488297 13004 ts_tablet_manager.cc:1397] T 228068c1c54e4e929e3153a366a30099 P 29842918acc045d29ea0fa4adfff5b9c: Time spent bootstrapping tablet: real 0.026s user 0.006s sys 0.004s
W20260924 00:21:47.490187 12990 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260924 00:21:47.490489 12990 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260924 00:21:47.490805 12990 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
I20260924 00:21:47.490576 13004 raft_consensus.cc:348] T 228068c1c54e4e929e3153a366a30099 P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
W20260924 00:21:47.491112 12990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
I20260924 00:21:47.491236 13004 raft_consensus.cc:374] T 228068c1c54e4e929e3153a366a30099 P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260924 00:21:47.491432 12990 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
I20260924 00:21:47.491472 13004 raft_consensus.cc:729] T 228068c1c54e4e929e3153a366a30099 P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29842918acc045d29ea0fa4adfff5b9c, State: Initialized, Role: FOLLOWER
W20260924 00:21:47.491750 12990 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:21:47.493525 13009 tablet_bootstrap.cc:492] T 228068c1c54e4e929e3153a366a30099 P fa329f7568e1416597742422ff0026cb: No bootstrap required, opened a new log
I20260924 00:21:47.493193 13004 consensus_queue.cc:260] T 228068c1c54e4e929e3153a366a30099 P 29842918acc045d29ea0fa4adfff5b9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:47.493966 13009 ts_tablet_manager.cc:1397] T 228068c1c54e4e929e3153a366a30099 P fa329f7568e1416597742422ff0026cb: Time spent bootstrapping tablet: real 0.041s user 0.005s sys 0.006s
I20260924 00:21:47.497539 13004 ts_tablet_manager.cc:1428] T 228068c1c54e4e929e3153a366a30099 P 29842918acc045d29ea0fa4adfff5b9c: Time spent starting tablet: real 0.009s user 0.006s sys 0.001s
I20260924 00:21:47.498332 13004 tablet_bootstrap.cc:492] T 198628c8cdac4c0dab683eeb55ae4a87 P 29842918acc045d29ea0fa4adfff5b9c: Bootstrap starting.
I20260924 00:21:47.498584 13008 tablet_bootstrap.cc:492] T 228068c1c54e4e929e3153a366a30099 P 08e4cf0195e645bcbb5ac5e3ede93096: No bootstrap required, opened a new log
I20260924 00:21:47.498956 13008 ts_tablet_manager.cc:1397] T 228068c1c54e4e929e3153a366a30099 P 08e4cf0195e645bcbb5ac5e3ede93096: Time spent bootstrapping tablet: real 0.033s user 0.005s sys 0.010s
I20260924 00:21:47.501294 13008 raft_consensus.cc:348] T 228068c1c54e4e929e3153a366a30099 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:47.501974 13008 raft_consensus.cc:374] T 228068c1c54e4e929e3153a366a30099 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:47.502245 13008 raft_consensus.cc:729] T 228068c1c54e4e929e3153a366a30099 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08e4cf0195e645bcbb5ac5e3ede93096, State: Initialized, Role: FOLLOWER
I20260924 00:21:47.502861 13008 consensus_queue.cc:260] T 228068c1c54e4e929e3153a366a30099 P 08e4cf0195e645bcbb5ac5e3ede93096 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:47.504805 13008 ts_tablet_manager.cc:1428] T 228068c1c54e4e929e3153a366a30099 P 08e4cf0195e645bcbb5ac5e3ede93096: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260924 00:21:47.505617 13008 tablet_bootstrap.cc:492] T 198628c8cdac4c0dab683eeb55ae4a87 P 08e4cf0195e645bcbb5ac5e3ede93096: Bootstrap starting.
I20260924 00:21:47.511572 12326 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33172:
name: "default.Table_e43f7024e9b745758c0915ae7605e671"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:47.521581 13004 tablet_bootstrap.cc:654] T 198628c8cdac4c0dab683eeb55ae4a87 P 29842918acc045d29ea0fa4adfff5b9c: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:47.524092 13008 tablet_bootstrap.cc:654] T 198628c8cdac4c0dab683eeb55ae4a87 P 08e4cf0195e645bcbb5ac5e3ede93096: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:47.531369 12317 master_service.cc:946] ReplaceTablet: received request to replace tablet 198628c8cdac4c0dab683eeb55ae4a87 from {username='slave'} at 127.0.0.1:33172
I20260924 00:21:47.531368 13009 raft_consensus.cc:348] T 228068c1c54e4e929e3153a366a30099 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:47.532049 13009 raft_consensus.cc:374] T 228068c1c54e4e929e3153a366a30099 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:47.532311 13009 raft_consensus.cc:729] T 228068c1c54e4e929e3153a366a30099 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa329f7568e1416597742422ff0026cb, State: Initialized, Role: FOLLOWER
@ 0x7f888209bf8a _ZNSt3__18__invokeIRZN4kudu6master30HmsNotificationLogListenerTask4InitEvE3$_0JEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOS6_DpOS7_ at ??:0
I20260924 00:21:47.533309 12319 master_service.cc:946] ReplaceTablet: received request to replace tablet 228068c1c54e4e929e3153a366a30099 from {username='slave'} at 127.0.0.1:33172
I20260924 00:21:47.537799 13009 consensus_queue.cc:260] T 228068c1c54e4e929e3153a366a30099 P fa329f7568e1416597742422ff0026cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:47.540884 13008 tablet_bootstrap.cc:492] T 198628c8cdac4c0dab683eeb55ae4a87 P 08e4cf0195e645bcbb5ac5e3ede93096: No bootstrap required, opened a new log
I20260924 00:21:47.541296 13008 ts_tablet_manager.cc:1397] T 198628c8cdac4c0dab683eeb55ae4a87 P 08e4cf0195e645bcbb5ac5e3ede93096: Time spent bootstrapping tablet: real 0.036s user 0.006s sys 0.004s
I20260924 00:21:47.545440 13008 raft_consensus.cc:348] T 198628c8cdac4c0dab683eeb55ae4a87 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } }
I20260924 00:21:47.546121 13008 raft_consensus.cc:374] T 198628c8cdac4c0dab683eeb55ae4a87 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:47.546141 13009 ts_tablet_manager.cc:1428] T 228068c1c54e4e929e3153a366a30099 P fa329f7568e1416597742422ff0026cb: Time spent starting tablet: real 0.052s user 0.004s sys 0.002s
I20260924 00:21:47.546358 13008 raft_consensus.cc:729] T 198628c8cdac4c0dab683eeb55ae4a87 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08e4cf0195e645bcbb5ac5e3ede93096, State: Initialized, Role: FOLLOWER
I20260924 00:21:47.549476 13008 consensus_queue.cc:260] T 198628c8cdac4c0dab683eeb55ae4a87 P 08e4cf0195e645bcbb5ac5e3ede93096 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } }
I20260924 00:21:47.551446 13008 ts_tablet_manager.cc:1428] T 198628c8cdac4c0dab683eeb55ae4a87 P 08e4cf0195e645bcbb5ac5e3ede93096: Time spent starting tablet: real 0.010s user 0.004s sys 0.000s
I20260924 00:21:47.563822 12317 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39: Sending DeleteTablet for 3 replicas of tablet 198628c8cdac4c0dab683eeb55ae4a87
I20260924 00:21:47.564054 13009 tablet_bootstrap.cc:492] T 198628c8cdac4c0dab683eeb55ae4a87 P fa329f7568e1416597742422ff0026cb: Bootstrap starting.
I20260924 00:21:47.565246 12317 catalog_manager.cc:6565] ReplaceTablet: tablet 198628c8cdac4c0dab683eeb55ae4a87 deleted and replaced by tablet 11ec8938396a435593baaca5e058677e
I20260924 00:21:47.566256 12879 tablet_service.cc:1514] Processing DeleteTablet for tablet 198628c8cdac4c0dab683eeb55ae4a87 with delete_type TABLET_DATA_DELETED (replaced by tablet 11ec8938396a435593baaca5e058677e) from {username='slave'} at 127.0.0.1:36822
I20260924 00:21:47.566933 13005 tablet_replica.cc:330] T 198628c8cdac4c0dab683eeb55ae4a87 P 08e4cf0195e645bcbb5ac5e3ede93096: stopping tablet replica
I20260924 00:21:47.567646 13005 raft_consensus.cc:2229] T 198628c8cdac4c0dab683eeb55ae4a87 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:47.567847 12735 tablet_service.cc:1514] Processing DeleteTablet for tablet 198628c8cdac4c0dab683eeb55ae4a87 with delete_type TABLET_DATA_DELETED (replaced by tablet 11ec8938396a435593baaca5e058677e) from {username='slave'} at 127.0.0.1:38968
I20260924 00:21:47.568066 13005 raft_consensus.cc:2258] T 198628c8cdac4c0dab683eeb55ae4a87 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:47.569161 12588 tablet_service.cc:1514] Processing DeleteTablet for tablet 198628c8cdac4c0dab683eeb55ae4a87 with delete_type TABLET_DATA_DELETED (replaced by tablet 11ec8938396a435593baaca5e058677e) from {username='slave'} at 127.0.0.1:33370
W20260924 00:21:47.570619 12310 catalog_manager.cc:4821] TS fa329f7568e1416597742422ff0026cb (127.10.4.66:40645): delete failed for tablet 198628c8cdac4c0dab683eeb55ae4a87 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 198628c8cdac4c0dab683eeb55ae4a87 already in progress: creating tablet
W20260924 00:21:47.570739 12313 catalog_manager.cc:4821] TS 29842918acc045d29ea0fa4adfff5b9c (127.10.4.65:46421): delete failed for tablet 198628c8cdac4c0dab683eeb55ae4a87 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 198628c8cdac4c0dab683eeb55ae4a87 already in progress: creating tablet
I20260924 00:21:47.571166 13005 ts_tablet_manager.cc:1905] T 198628c8cdac4c0dab683eeb55ae4a87 P 08e4cf0195e645bcbb5ac5e3ede93096: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7f888209bf1a std::__1::__invoke_void_return_wrapper<>::__call<>() at ??:0
W20260924 00:21:47.571481 12660 tablet.cc:2369] T 1c3ffccda42840d88b039e39b9ecc2c7 P 29842918acc045d29ea0fa4adfff5b9c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:21:47.579762 13004 tablet_bootstrap.cc:492] T 198628c8cdac4c0dab683eeb55ae4a87 P 29842918acc045d29ea0fa4adfff5b9c: No bootstrap required, opened a new log
I20260924 00:21:47.580169 13004 ts_tablet_manager.cc:1397] T 198628c8cdac4c0dab683eeb55ae4a87 P 29842918acc045d29ea0fa4adfff5b9c: Time spent bootstrapping tablet: real 0.082s user 0.006s sys 0.006s
I20260924 00:21:47.582636 13004 raft_consensus.cc:348] T 198628c8cdac4c0dab683eeb55ae4a87 P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } }
I20260924 00:21:47.583293 13004 raft_consensus.cc:374] T 198628c8cdac4c0dab683eeb55ae4a87 P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:47.583205 12990 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.4.84:44801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:45439
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.10.4.126:37913,127.10.4.125:46875,127.10.4.124:45407
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.4.124:45407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.4.124
--webserver_port=36597
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:32:10 UTC on ccycloud.quasar-wfxeao.root.comops.site
build id 4668917
TSAN enabled
I20260924 00:21:47.583561 13004 raft_consensus.cc:729] T 198628c8cdac4c0dab683eeb55ae4a87 P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29842918acc045d29ea0fa4adfff5b9c, State: Initialized, Role: FOLLOWER
I20260924 00:21:47.584244 13004 consensus_queue.cc:260] T 198628c8cdac4c0dab683eeb55ae4a87 P 29842918acc045d29ea0fa4adfff5b9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } }
I20260924 00:21:47.584719 12990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:21:47.586805 12990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
@ 0x7f888209bee2 std::__1::__function::__alloc_func<>::operator()() at ??:0
I20260924 00:21:47.592917 13004 ts_tablet_manager.cc:1428] T 198628c8cdac4c0dab683eeb55ae4a87 P 29842918acc045d29ea0fa4adfff5b9c: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
I20260924 00:21:47.595650 12735 tablet_service.cc:1514] Processing DeleteTablet for tablet 228068c1c54e4e929e3153a366a30099 with delete_type TABLET_DATA_DELETED (replaced by tablet 71c636b9c7c24cfcaae4ca0596a45ae6) from {username='slave'} at 127.0.0.1:38968
I20260924 00:21:47.596326 13006 tablet_replica.cc:330] T 228068c1c54e4e929e3153a366a30099 P fa329f7568e1416597742422ff0026cb: stopping tablet replica
I20260924 00:21:47.597050 13006 raft_consensus.cc:2229] T 228068c1c54e4e929e3153a366a30099 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:47.597471 13006 raft_consensus.cc:2258] T 228068c1c54e4e929e3153a366a30099 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:47.599531 13006 ts_tablet_manager.cc:1905] T 228068c1c54e4e929e3153a366a30099 P fa329f7568e1416597742422ff0026cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:47.600754 12319 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fb563a88aa00432ba12db382b1435a39: Sending DeleteTablet for 3 replicas of tablet 228068c1c54e4e929e3153a366a30099
I20260924 00:21:47.600831 12588 tablet_service.cc:1514] Processing DeleteTablet for tablet 228068c1c54e4e929e3153a366a30099 with delete_type TABLET_DATA_DELETED (replaced by tablet 71c636b9c7c24cfcaae4ca0596a45ae6) from {username='slave'} at 127.0.0.1:33370
I20260924 00:21:47.601734 13007 tablet_replica.cc:330] T 228068c1c54e4e929e3153a366a30099 P 29842918acc045d29ea0fa4adfff5b9c: stopping tablet replica
I20260924 00:21:47.602320 12588 tablet_service.cc:1514] Processing DeleteTablet for tablet 228068c1c54e4e929e3153a366a30099 with delete_type TABLET_DATA_DELETED (replaced by tablet 71c636b9c7c24cfcaae4ca0596a45ae6) from {username='slave'} at 127.0.0.1:33370
I20260924 00:21:47.602432 13007 raft_consensus.cc:2229] T 228068c1c54e4e929e3153a366a30099 P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:47.602728 12735 tablet_service.cc:1514] Processing DeleteTablet for tablet 228068c1c54e4e929e3153a366a30099 with delete_type TABLET_DATA_DELETED (replaced by tablet 71c636b9c7c24cfcaae4ca0596a45ae6) from {username='slave'} at 127.0.0.1:38968
I20260924 00:21:47.602883 13007 raft_consensus.cc:2258] T 228068c1c54e4e929e3153a366a30099 P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:47.604318 12319 catalog_manager.cc:6565] ReplaceTablet: tablet 228068c1c54e4e929e3153a366a30099 deleted and replaced by tablet 71c636b9c7c24cfcaae4ca0596a45ae6
I20260924 00:21:47.604858 13007 ts_tablet_manager.cc:1905] T 228068c1c54e4e929e3153a366a30099 P 29842918acc045d29ea0fa4adfff5b9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:47.605278 12879 tablet_service.cc:1514] Processing DeleteTablet for tablet 228068c1c54e4e929e3153a366a30099 with delete_type TABLET_DATA_DELETED (replaced by tablet 71c636b9c7c24cfcaae4ca0596a45ae6) from {username='slave'} at 127.0.0.1:36822
W20260924 00:21:47.614794 12803 tablet.cc:2369] T 1c3ffccda42840d88b039e39b9ecc2c7 P fa329f7568e1416597742422ff0026cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:21:47.618505 13007 ts_tablet_manager.cc:1918] T 228068c1c54e4e929e3153a366a30099 P 29842918acc045d29ea0fa4adfff5b9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:47.618858 13007 log.cc:1196] T 228068c1c54e4e929e3153a366a30099 P 29842918acc045d29ea0fa4adfff5b9c: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/228068c1c54e4e929e3153a366a30099
@ 0x7f888209b1de std::__1::__function::__func<>::operator()() at ??:0
@ 0x580785 std::__1::__function::__value_func<>::operator()() at ??:?
I20260924 00:21:47.629727 13007 ts_tablet_manager.cc:1939] T 228068c1c54e4e929e3153a366a30099 P 29842918acc045d29ea0fa4adfff5b9c: Deleting consensus metadata
I20260924 00:21:47.632133 12313 catalog_manager.cc:4841] TS 29842918acc045d29ea0fa4adfff5b9c (127.10.4.65:46421): tablet 228068c1c54e4e929e3153a366a30099 (table default.table_027aad11c8d7454bbf92402722920c2a [id=291e8d50dc5248c280e4fde27a818707]) successfully deleted
I20260924 00:21:47.633940 12588 tablet_service.cc:1514] Processing DeleteTablet for tablet 198628c8cdac4c0dab683eeb55ae4a87 with delete_type TABLET_DATA_DELETED (replaced by tablet 11ec8938396a435593baaca5e058677e) from {username='slave'} at 127.0.0.1:33370
I20260924 00:21:47.639634 13009 tablet_bootstrap.cc:654] T 198628c8cdac4c0dab683eeb55ae4a87 P fa329f7568e1416597742422ff0026cb: Neither blocks nor log segments found. Creating new log.
W20260924 00:21:47.643034 12313 catalog_manager.cc:4808] TS 29842918acc045d29ea0fa4adfff5b9c (127.10.4.65:46421): delete failed for tablet 228068c1c54e4e929e3153a366a30099 because the tablet was not found. No further retry: Not found: Tablet not found: 228068c1c54e4e929e3153a366a30099
I20260924 00:21:47.644004 13007 tablet_replica.cc:330] T 198628c8cdac4c0dab683eeb55ae4a87 P 29842918acc045d29ea0fa4adfff5b9c: stopping tablet replica
@ 0x5805ba std::__1::function<>::operator()() at ??:?
I20260924 00:21:47.644658 13007 raft_consensus.cc:2229] T 198628c8cdac4c0dab683eeb55ae4a87 P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:47.645102 13007 raft_consensus.cc:2258] T 198628c8cdac4c0dab683eeb55ae4a87 P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:47.650604 13005 ts_tablet_manager.cc:1918] T 198628c8cdac4c0dab683eeb55ae4a87 P 08e4cf0195e645bcbb5ac5e3ede93096: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:47.650956 13005 log.cc:1196] T 198628c8cdac4c0dab683eeb55ae4a87 P 08e4cf0195e645bcbb5ac5e3ede93096: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/198628c8cdac4c0dab683eeb55ae4a87
I20260924 00:21:47.651566 13005 ts_tablet_manager.cc:1939] T 198628c8cdac4c0dab683eeb55ae4a87 P 08e4cf0195e645bcbb5ac5e3ede93096: Deleting consensus metadata
I20260924 00:21:47.655367 13007 ts_tablet_manager.cc:1905] T 198628c8cdac4c0dab683eeb55ae4a87 P 29842918acc045d29ea0fa4adfff5b9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:47.656335 13006 ts_tablet_manager.cc:1918] T 228068c1c54e4e929e3153a366a30099 P fa329f7568e1416597742422ff0026cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:47.656700 13006 log.cc:1196] T 228068c1c54e4e929e3153a366a30099 P fa329f7568e1416597742422ff0026cb: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/228068c1c54e4e929e3153a366a30099
I20260924 00:21:47.656862 12735 tablet_service.cc:1467] Processing CreateTablet for tablet 11ec8938396a435593baaca5e058677e (DEFAULT_TABLE table=default.table_6888b4491c68473c8fbc8622820afc42 [id=670fab2a44eb41ee89d1a16ef0aef90c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:47.657300 13006 ts_tablet_manager.cc:1939] T 228068c1c54e4e929e3153a366a30099 P fa329f7568e1416597742422ff0026cb: Deleting consensus metadata
I20260924 00:21:47.657876 12735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11ec8938396a435593baaca5e058677e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:47.657971 12588 tablet_service.cc:1467] Processing CreateTablet for tablet 11ec8938396a435593baaca5e058677e (DEFAULT_TABLE table=default.table_6888b4491c68473c8fbc8622820afc42 [id=670fab2a44eb41ee89d1a16ef0aef90c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:47.658967 12588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11ec8938396a435593baaca5e058677e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:47.658996 12879 tablet_service.cc:1467] Processing CreateTablet for tablet 11ec8938396a435593baaca5e058677e (DEFAULT_TABLE table=default.table_6888b4491c68473c8fbc8622820afc42 [id=670fab2a44eb41ee89d1a16ef0aef90c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:47.659799 13005 tablet_replica.cc:330] T 228068c1c54e4e929e3153a366a30099 P 08e4cf0195e645bcbb5ac5e3ede93096: stopping tablet replica
I20260924 00:21:47.659988 12879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11ec8938396a435593baaca5e058677e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:47.660462 13005 raft_consensus.cc:2229] T 228068c1c54e4e929e3153a366a30099 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:47.662495 12310 catalog_manager.cc:4841] TS fa329f7568e1416597742422ff0026cb (127.10.4.66:40645): tablet 228068c1c54e4e929e3153a366a30099 (table default.table_027aad11c8d7454bbf92402722920c2a [id=291e8d50dc5248c280e4fde27a818707]) successfully deleted
I20260924 00:21:47.663372 12313 catalog_manager.cc:4841] TS 08e4cf0195e645bcbb5ac5e3ede93096 (127.10.4.67:38139): tablet 198628c8cdac4c0dab683eeb55ae4a87 (table default.table_6888b4491c68473c8fbc8622820afc42 [id=670fab2a44eb41ee89d1a16ef0aef90c]) successfully deleted
W20260924 00:21:47.664645 12310 catalog_manager.cc:4808] TS fa329f7568e1416597742422ff0026cb (127.10.4.66:40645): delete failed for tablet 228068c1c54e4e929e3153a366a30099 because the tablet was not found. No further retry: Not found: Tablet not found: 228068c1c54e4e929e3153a366a30099
I20260924 00:21:47.660866 13005 raft_consensus.cc:2258] T 228068c1c54e4e929e3153a366a30099 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:47.673585 13005 ts_tablet_manager.cc:1905] T 228068c1c54e4e929e3153a366a30099 P 08e4cf0195e645bcbb5ac5e3ede93096: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:47.674559 13015 raft_consensus.cc:482] T 1c3ffccda42840d88b039e39b9ecc2c7 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:47.675088 12587 tablet_service.cc:1467] Processing CreateTablet for tablet 71c636b9c7c24cfcaae4ca0596a45ae6 (DEFAULT_TABLE table=default.table_027aad11c8d7454bbf92402722920c2a [id=291e8d50dc5248c280e4fde27a818707]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:47.675066 13015 raft_consensus.cc:504] T 1c3ffccda42840d88b039e39b9ecc2c7 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } }
I20260924 00:21:47.676105 12587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71c636b9c7c24cfcaae4ca0596a45ae6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:47.681226 13015 leader_election.cc:288] T 1c3ffccda42840d88b039e39b9ecc2c7 P fa329f7568e1416597742422ff0026cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08e4cf0195e645bcbb5ac5e3ede93096 (127.10.4.67:38139), 29842918acc045d29ea0fa4adfff5b9c (127.10.4.65:46421)
I20260924 00:21:47.683709 13004 tablet_bootstrap.cc:492] T 11ec8938396a435593baaca5e058677e P 29842918acc045d29ea0fa4adfff5b9c: Bootstrap starting.
I20260924 00:21:47.691399 12735 tablet_service.cc:1467] Processing CreateTablet for tablet 71c636b9c7c24cfcaae4ca0596a45ae6 (DEFAULT_TABLE table=default.table_027aad11c8d7454bbf92402722920c2a [id=291e8d50dc5248c280e4fde27a818707]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:47.692497 12735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71c636b9c7c24cfcaae4ca0596a45ae6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:47.699621 13008 tablet_bootstrap.cc:492] T 11ec8938396a435593baaca5e058677e P 08e4cf0195e645bcbb5ac5e3ede93096: Bootstrap starting.
I20260924 00:21:47.702149 13004 tablet_bootstrap.cc:654] T 11ec8938396a435593baaca5e058677e P 29842918acc045d29ea0fa4adfff5b9c: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:47.705660 13008 tablet_bootstrap.cc:654] T 11ec8938396a435593baaca5e058677e P 08e4cf0195e645bcbb5ac5e3ede93096: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:47.716251 13008 tablet_bootstrap.cc:492] T 11ec8938396a435593baaca5e058677e P 08e4cf0195e645bcbb5ac5e3ede93096: No bootstrap required, opened a new log
I20260924 00:21:47.716626 13008 ts_tablet_manager.cc:1397] T 11ec8938396a435593baaca5e058677e P 08e4cf0195e645bcbb5ac5e3ede93096: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.000s
I20260924 00:21:47.718456 13009 tablet_bootstrap.cc:492] T 198628c8cdac4c0dab683eeb55ae4a87 P fa329f7568e1416597742422ff0026cb: No bootstrap required, opened a new log
I20260924 00:21:47.718842 13009 ts_tablet_manager.cc:1397] T 198628c8cdac4c0dab683eeb55ae4a87 P fa329f7568e1416597742422ff0026cb: Time spent bootstrapping tablet: real 0.155s user 0.009s sys 0.001s
I20260924 00:21:47.719384 13008 raft_consensus.cc:348] T 11ec8938396a435593baaca5e058677e P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:47.720132 13008 raft_consensus.cc:374] T 11ec8938396a435593baaca5e058677e P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:47.720363 13008 raft_consensus.cc:729] T 11ec8938396a435593baaca5e058677e P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08e4cf0195e645bcbb5ac5e3ede93096, State: Initialized, Role: FOLLOWER
I20260924 00:21:47.720980 13008 consensus_queue.cc:260] T 11ec8938396a435593baaca5e058677e P 08e4cf0195e645bcbb5ac5e3ede93096 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
W20260924 00:21:47.723951 13048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:21:47.726003 13007 ts_tablet_manager.cc:1918] T 198628c8cdac4c0dab683eeb55ae4a87 P 29842918acc045d29ea0fa4adfff5b9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:47.726368 13007 log.cc:1196] T 198628c8cdac4c0dab683eeb55ae4a87 P 29842918acc045d29ea0fa4adfff5b9c: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/198628c8cdac4c0dab683eeb55ae4a87
I20260924 00:21:47.726472 13008 ts_tablet_manager.cc:1428] T 11ec8938396a435593baaca5e058677e P 08e4cf0195e645bcbb5ac5e3ede93096: Time spent starting tablet: real 0.009s user 0.004s sys 0.000s
I20260924 00:21:47.726958 13007 ts_tablet_manager.cc:1939] T 198628c8cdac4c0dab683eeb55ae4a87 P 29842918acc045d29ea0fa4adfff5b9c: Deleting consensus metadata
I20260924 00:21:47.716733 13005 ts_tablet_manager.cc:1918] T 228068c1c54e4e929e3153a366a30099 P 08e4cf0195e645bcbb5ac5e3ede93096: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:47.734524 12313 catalog_manager.cc:4841] TS 29842918acc045d29ea0fa4adfff5b9c (127.10.4.65:46421): tablet 198628c8cdac4c0dab683eeb55ae4a87 (table default.table_6888b4491c68473c8fbc8622820afc42 [id=670fab2a44eb41ee89d1a16ef0aef90c]) successfully deleted
I20260924 00:21:47.737511 13013 raft_consensus.cc:482] T 1c3ffccda42840d88b039e39b9ecc2c7 P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:47.738039 13013 raft_consensus.cc:504] T 1c3ffccda42840d88b039e39b9ecc2c7 P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } }
I20260924 00:21:47.729707 13005 log.cc:1196] T 228068c1c54e4e929e3153a366a30099 P 08e4cf0195e645bcbb5ac5e3ede93096: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/228068c1c54e4e929e3153a366a30099
I20260924 00:21:47.743609 13005 ts_tablet_manager.cc:1939] T 228068c1c54e4e929e3153a366a30099 P 08e4cf0195e645bcbb5ac5e3ede93096: Deleting consensus metadata
I20260924 00:21:47.745611 13013 leader_election.cc:288] T 1c3ffccda42840d88b039e39b9ecc2c7 P 29842918acc045d29ea0fa4adfff5b9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08e4cf0195e645bcbb5ac5e3ede93096 (127.10.4.67:38139), fa329f7568e1416597742422ff0026cb (127.10.4.66:40645)
@ 0x7f88791aedc7 kudu::Thread::SuperviseThread() at ??:0
@ 0x4230f0 __tsan_thread_start_func at ??:?
I20260924 00:21:47.775969 12319 master_service.cc:946] ReplaceTablet: received request to replace tablet 71c636b9c7c24cfcaae4ca0596a45ae6 from {username='slave'} at 127.0.0.1:33172
I20260924 00:21:47.776811 13004 tablet_bootstrap.cc:492] T 11ec8938396a435593baaca5e058677e P 29842918acc045d29ea0fa4adfff5b9c: No bootstrap required, opened a new log
I20260924 00:21:47.777249 13004 ts_tablet_manager.cc:1397] T 11ec8938396a435593baaca5e058677e P 29842918acc045d29ea0fa4adfff5b9c: Time spent bootstrapping tablet: real 0.094s user 0.012s sys 0.000s
I20260924 00:21:47.781126 12313 catalog_manager.cc:4841] TS 08e4cf0195e645bcbb5ac5e3ede93096 (127.10.4.67:38139): tablet 228068c1c54e4e929e3153a366a30099 (table default.table_027aad11c8d7454bbf92402722920c2a [id=291e8d50dc5248c280e4fde27a818707]) successfully deleted
I20260924 00:21:47.783143 13009 raft_consensus.cc:348] T 198628c8cdac4c0dab683eeb55ae4a87 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } }
I20260924 00:21:47.783823 13009 raft_consensus.cc:374] T 198628c8cdac4c0dab683eeb55ae4a87 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:47.784072 13009 raft_consensus.cc:729] T 198628c8cdac4c0dab683eeb55ae4a87 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa329f7568e1416597742422ff0026cb, State: Initialized, Role: FOLLOWER
I20260924 00:21:47.784041 12879 tablet_service.cc:1467] Processing CreateTablet for tablet 71c636b9c7c24cfcaae4ca0596a45ae6 (DEFAULT_TABLE table=default.table_027aad11c8d7454bbf92402722920c2a [id=291e8d50dc5248c280e4fde27a818707]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:47.785125 12879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71c636b9c7c24cfcaae4ca0596a45ae6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:47.784672 13009 consensus_queue.cc:260] T 198628c8cdac4c0dab683eeb55ae4a87 P fa329f7568e1416597742422ff0026cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } }
I20260924 00:21:47.779435 13004 raft_consensus.cc:348] T 11ec8938396a435593baaca5e058677e P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:47.788404 13004 raft_consensus.cc:374] T 11ec8938396a435593baaca5e058677e P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:47.788661 13004 raft_consensus.cc:729] T 11ec8938396a435593baaca5e058677e P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29842918acc045d29ea0fa4adfff5b9c, State: Initialized, Role: FOLLOWER
@ 0x7f887a81c6db start_thread at ??:0
I20260924 00:21:47.795475 13009 ts_tablet_manager.cc:1428] T 198628c8cdac4c0dab683eeb55ae4a87 P fa329f7568e1416597742422ff0026cb: Time spent starting tablet: real 0.076s user 0.003s sys 0.004s
I20260924 00:21:47.796330 13009 tablet_bootstrap.cc:492] T 11ec8938396a435593baaca5e058677e P fa329f7568e1416597742422ff0026cb: Bootstrap starting.
I20260924 00:21:47.797669 12735 tablet_service.cc:1514] Processing DeleteTablet for tablet 198628c8cdac4c0dab683eeb55ae4a87 with delete_type TABLET_DATA_DELETED (replaced by tablet 11ec8938396a435593baaca5e058677e) from {username='slave'} at 127.0.0.1:38968
I20260924 00:21:47.797338 13004 consensus_queue.cc:260] T 11ec8938396a435593baaca5e058677e P 29842918acc045d29ea0fa4adfff5b9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
@ 0x7f8875ca261f clone at ??:0
I20260924 00:21:47.806234 13006 tablet_replica.cc:330] T 198628c8cdac4c0dab683eeb55ae4a87 P fa329f7568e1416597742422ff0026cb: stopping tablet replica
I20260924 00:21:47.806957 13006 raft_consensus.cc:2229] T 198628c8cdac4c0dab683eeb55ae4a87 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:47.807512 13006 raft_consensus.cc:2258] T 198628c8cdac4c0dab683eeb55ae4a87 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:47.810817 13004 ts_tablet_manager.cc:1428] T 11ec8938396a435593baaca5e058677e P 29842918acc045d29ea0fa4adfff5b9c: Time spent starting tablet: real 0.033s user 0.006s sys 0.000s
I20260924 00:21:47.811679 13004 tablet_bootstrap.cc:492] T 71c636b9c7c24cfcaae4ca0596a45ae6 P 29842918acc045d29ea0fa4adfff5b9c: Bootstrap starting.
I20260924 00:21:47.817912 13006 ts_tablet_manager.cc:1905] T 198628c8cdac4c0dab683eeb55ae4a87 P fa329f7568e1416597742422ff0026cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:47.834095 13009 tablet_bootstrap.cc:654] T 11ec8938396a435593baaca5e058677e P fa329f7568e1416597742422ff0026cb: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:47.838773 13004 tablet_bootstrap.cc:654] T 71c636b9c7c24cfcaae4ca0596a45ae6 P 29842918acc045d29ea0fa4adfff5b9c: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:47.844643 13009 tablet_bootstrap.cc:492] T 11ec8938396a435593baaca5e058677e P fa329f7568e1416597742422ff0026cb: No bootstrap required, opened a new log
I20260924 00:21:47.845117 13009 ts_tablet_manager.cc:1397] T 11ec8938396a435593baaca5e058677e P fa329f7568e1416597742422ff0026cb: Time spent bootstrapping tablet: real 0.049s user 0.010s sys 0.000s
I20260924 00:21:47.847541 13009 raft_consensus.cc:348] T 11ec8938396a435593baaca5e058677e P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:47.848197 13009 raft_consensus.cc:374] T 11ec8938396a435593baaca5e058677e P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:47.848417 13009 raft_consensus.cc:729] T 11ec8938396a435593baaca5e058677e P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa329f7568e1416597742422ff0026cb, State: Initialized, Role: FOLLOWER
I20260924 00:21:47.849742 13009 consensus_queue.cc:260] T 11ec8938396a435593baaca5e058677e P fa329f7568e1416597742422ff0026cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:47.853948 13009 ts_tablet_manager.cc:1428] T 11ec8938396a435593baaca5e058677e P fa329f7568e1416597742422ff0026cb: Time spent starting tablet: real 0.009s user 0.005s sys 0.001s
I20260924 00:21:47.858165 13009 tablet_bootstrap.cc:492] T 71c636b9c7c24cfcaae4ca0596a45ae6 P fa329f7568e1416597742422ff0026cb: Bootstrap starting.
I20260924 00:21:47.861452 13004 tablet_bootstrap.cc:492] T 71c636b9c7c24cfcaae4ca0596a45ae6 P 29842918acc045d29ea0fa4adfff5b9c: No bootstrap required, opened a new log
I20260924 00:21:47.861840 13004 ts_tablet_manager.cc:1397] T 71c636b9c7c24cfcaae4ca0596a45ae6 P 29842918acc045d29ea0fa4adfff5b9c: Time spent bootstrapping tablet: real 0.050s user 0.011s sys 0.007s
I20260924 00:21:47.863849 13008 tablet_bootstrap.cc:492] T 71c636b9c7c24cfcaae4ca0596a45ae6 P 08e4cf0195e645bcbb5ac5e3ede93096: Bootstrap starting.
I20260924 00:21:47.863952 13004 raft_consensus.cc:348] T 71c636b9c7c24cfcaae4ca0596a45ae6 P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:47.864619 13004 raft_consensus.cc:374] T 71c636b9c7c24cfcaae4ca0596a45ae6 P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:47.864854 13004 raft_consensus.cc:729] T 71c636b9c7c24cfcaae4ca0596a45ae6 P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29842918acc045d29ea0fa4adfff5b9c, State: Initialized, Role: FOLLOWER
I20260924 00:21:47.865650 13004 consensus_queue.cc:260] T 71c636b9c7c24cfcaae4ca0596a45ae6 P 29842918acc045d29ea0fa4adfff5b9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:47.867879 13004 ts_tablet_manager.cc:1428] T 71c636b9c7c24cfcaae4ca0596a45ae6 P 29842918acc045d29ea0fa4adfff5b9c: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260924 00:21:47.870676 13009 tablet_bootstrap.cc:654] T 71c636b9c7c24cfcaae4ca0596a45ae6 P fa329f7568e1416597742422ff0026cb: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:47.871091 13008 tablet_bootstrap.cc:654] T 71c636b9c7c24cfcaae4ca0596a45ae6 P 08e4cf0195e645bcbb5ac5e3ede93096: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:47.872967 13006 ts_tablet_manager.cc:1918] T 198628c8cdac4c0dab683eeb55ae4a87 P fa329f7568e1416597742422ff0026cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 00:21:47.890820 12608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c3ffccda42840d88b039e39b9ecc2c7" candidate_uuid: "fa329f7568e1416597742422ff0026cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29842918acc045d29ea0fa4adfff5b9c" is_pre_election: true
I20260924 00:21:47.891691 12608 raft_consensus.cc:2454] T 1c3ffccda42840d88b039e39b9ecc2c7 P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa329f7568e1416597742422ff0026cb in term 0.
I20260924 00:21:47.893363 12691 leader_election.cc:302] T 1c3ffccda42840d88b039e39b9ecc2c7 P fa329f7568e1416597742422ff0026cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29842918acc045d29ea0fa4adfff5b9c, fa329f7568e1416597742422ff0026cb; no voters:
I20260924 00:21:47.894273 13015 raft_consensus.cc:2781] T 1c3ffccda42840d88b039e39b9ecc2c7 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:47.894632 13015 raft_consensus.cc:482] T 1c3ffccda42840d88b039e39b9ecc2c7 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:47.894971 13015 raft_consensus.cc:3037] T 1c3ffccda42840d88b039e39b9ecc2c7 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:47.901619 13015 raft_consensus.cc:504] T 1c3ffccda42840d88b039e39b9ecc2c7 P fa329f7568e1416597742422ff0026cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } }
I20260924 00:21:47.904816 12608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c3ffccda42840d88b039e39b9ecc2c7" candidate_uuid: "fa329f7568e1416597742422ff0026cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29842918acc045d29ea0fa4adfff5b9c"
I20260924 00:21:47.905356 12608 raft_consensus.cc:3037] T 1c3ffccda42840d88b039e39b9ecc2c7 P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:47.905745 13006 log.cc:1196] T 198628c8cdac4c0dab683eeb55ae4a87 P fa329f7568e1416597742422ff0026cb: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/198628c8cdac4c0dab683eeb55ae4a87
I20260924 00:21:47.906410 13006 ts_tablet_manager.cc:1939] T 198628c8cdac4c0dab683eeb55ae4a87 P fa329f7568e1416597742422ff0026cb: Deleting consensus metadata
I20260924 00:21:47.911958 12608 raft_consensus.cc:2454] T 1c3ffccda42840d88b039e39b9ecc2c7 P 29842918acc045d29ea0fa4adfff5b9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa329f7568e1416597742422ff0026cb in term 1.
I20260924 00:21:47.912897 12691 leader_election.cc:302] T 1c3ffccda42840d88b039e39b9ecc2c7 P fa329f7568e1416597742422ff0026cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29842918acc045d29ea0fa4adfff5b9c, fa329f7568e1416597742422ff0026cb; no voters:
I20260924 00:21:47.914342 13015 leader_election.cc:288] T 1c3ffccda42840d88b039e39b9ecc2c7 P fa329f7568e1416597742422ff0026cb [CANDIDATE]: Term 1 election: Requested vote from peers 08e4cf0195e645bcbb5ac5e3ede93096 (127.10.4.67:38139), 29842918acc045d29ea0fa4adfff5b9c (127.10.4.65:46421)
I20260924 00:21:47.914521 13067 raft_consensus.cc:2781] T 1c3ffccda42840d88b039e39b9ecc2c7 P fa329f7568e1416597742422ff0026cb [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:47.915050 13067 raft_consensus.cc:686] T 1c3ffccda42840d88b039e39b9ecc2c7 P fa329f7568e1416597742422ff0026cb [term 1 LEADER]: Becoming Leader. State: Replica: fa329f7568e1416597742422ff0026cb, State: Running, Role: LEADER
I20260924 00:21:47.919163 13067 consensus_queue.cc:237] T 1c3ffccda42840d88b039e39b9ecc2c7 P fa329f7568e1416597742422ff0026cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } }
I20260924 00:21:47.923808 12755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c3ffccda42840d88b039e39b9ecc2c7" candidate_uuid: "29842918acc045d29ea0fa4adfff5b9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa329f7568e1416597742422ff0026cb" is_pre_election: true
I20260924 00:21:47.930567 12899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c3ffccda42840d88b039e39b9ecc2c7" candidate_uuid: "29842918acc045d29ea0fa4adfff5b9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" is_pre_election: true
I20260924 00:21:47.931329 12899 raft_consensus.cc:2454] T 1c3ffccda42840d88b039e39b9ecc2c7 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29842918acc045d29ea0fa4adfff5b9c in term 0.
I20260924 00:21:47.932503 12544 leader_election.cc:302] T 1c3ffccda42840d88b039e39b9ecc2c7 P 29842918acc045d29ea0fa4adfff5b9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 08e4cf0195e645bcbb5ac5e3ede93096, 29842918acc045d29ea0fa4adfff5b9c; no voters: fa329f7568e1416597742422ff0026cb
I20260924 00:21:47.933573 13013 raft_consensus.cc:2741] T 1c3ffccda42840d88b039e39b9ecc2c7 P 29842918acc045d29ea0fa4adfff5b9c [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260924 00:21:47.935613 13008 tablet_bootstrap.cc:492] T 71c636b9c7c24cfcaae4ca0596a45ae6 P 08e4cf0195e645bcbb5ac5e3ede93096: No bootstrap required, opened a new log
I20260924 00:21:47.935928 13008 ts_tablet_manager.cc:1397] T 71c636b9c7c24cfcaae4ca0596a45ae6 P 08e4cf0195e645bcbb5ac5e3ede93096: Time spent bootstrapping tablet: real 0.072s user 0.004s sys 0.004s
I20260924 00:21:47.941457 12898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c3ffccda42840d88b039e39b9ecc2c7" candidate_uuid: "fa329f7568e1416597742422ff0026cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08e4cf0195e645bcbb5ac5e3ede93096"
I20260924 00:21:47.942063 12898 raft_consensus.cc:3037] T 1c3ffccda42840d88b039e39b9ecc2c7 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:47.945338 12899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c3ffccda42840d88b039e39b9ecc2c7" candidate_uuid: "fa329f7568e1416597742422ff0026cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" is_pre_election: true
I20260924 00:21:47.950591 12898 raft_consensus.cc:2454] T 1c3ffccda42840d88b039e39b9ecc2c7 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa329f7568e1416597742422ff0026cb in term 1.
I20260924 00:21:47.953437 13008 raft_consensus.cc:348] T 71c636b9c7c24cfcaae4ca0596a45ae6 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:47.954016 13008 raft_consensus.cc:374] T 71c636b9c7c24cfcaae4ca0596a45ae6 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:47.955014 13008 raft_consensus.cc:729] T 71c636b9c7c24cfcaae4ca0596a45ae6 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08e4cf0195e645bcbb5ac5e3ede93096, State: Initialized, Role: FOLLOWER
I20260924 00:21:47.955641 13008 consensus_queue.cc:260] T 71c636b9c7c24cfcaae4ca0596a45ae6 P 08e4cf0195e645bcbb5ac5e3ede93096 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:47.962814 13009 tablet_bootstrap.cc:492] T 71c636b9c7c24cfcaae4ca0596a45ae6 P fa329f7568e1416597742422ff0026cb: No bootstrap required, opened a new log
I20260924 00:21:47.963208 13009 ts_tablet_manager.cc:1397] T 71c636b9c7c24cfcaae4ca0596a45ae6 P fa329f7568e1416597742422ff0026cb: Time spent bootstrapping tablet: real 0.105s user 0.026s sys 0.066s
I20260924 00:21:47.966979 13008 ts_tablet_manager.cc:1428] T 71c636b9c7c24cfcaae4ca0596a45ae6 P 08e4cf0195e645bcbb5ac5e3ede93096: Time spent starting tablet: real 0.031s user 0.006s sys 0.000s
I20260924 00:21:47.965497 13009 raft_consensus.cc:348] T 71c636b9c7c24cfcaae4ca0596a45ae6 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:47.967309 13009 raft_consensus.cc:374] T 71c636b9c7c24cfcaae4ca0596a45ae6 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:47.967546 13009 raft_consensus.cc:729] T 71c636b9c7c24cfcaae4ca0596a45ae6 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa329f7568e1416597742422ff0026cb, State: Initialized, Role: FOLLOWER
I20260924 00:21:47.973241 13009 consensus_queue.cc:260] T 71c636b9c7c24cfcaae4ca0596a45ae6 P fa329f7568e1416597742422ff0026cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:47.975111 13009 ts_tablet_manager.cc:1428] T 71c636b9c7c24cfcaae4ca0596a45ae6 P fa329f7568e1416597742422ff0026cb: Time spent starting tablet: real 0.012s user 0.005s sys 0.001s
I20260924 00:21:48.014629 13016 raft_consensus.cc:482] T 11ec8938396a435593baaca5e058677e P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:48.015081 13016 raft_consensus.cc:504] T 11ec8938396a435593baaca5e058677e P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:48.031451 13016 leader_election.cc:288] T 11ec8938396a435593baaca5e058677e P 08e4cf0195e645bcbb5ac5e3ede93096 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa329f7568e1416597742422ff0026cb (127.10.4.66:40645), 29842918acc045d29ea0fa4adfff5b9c (127.10.4.65:46421)
I20260924 00:21:48.034902 12755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11ec8938396a435593baaca5e058677e" candidate_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa329f7568e1416597742422ff0026cb" is_pre_election: true
I20260924 00:21:48.035516 12755 raft_consensus.cc:2454] T 11ec8938396a435593baaca5e058677e P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08e4cf0195e645bcbb5ac5e3ede93096 in term 0.
I20260924 00:21:48.036743 12832 leader_election.cc:302] T 11ec8938396a435593baaca5e058677e P 08e4cf0195e645bcbb5ac5e3ede93096 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08e4cf0195e645bcbb5ac5e3ede93096, fa329f7568e1416597742422ff0026cb; no voters:
I20260924 00:21:48.037667 13016 raft_consensus.cc:2781] T 11ec8938396a435593baaca5e058677e P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:48.038060 13016 raft_consensus.cc:482] T 11ec8938396a435593baaca5e058677e P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:48.038437 13016 raft_consensus.cc:3037] T 11ec8938396a435593baaca5e058677e P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:48.054895 12608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11ec8938396a435593baaca5e058677e" candidate_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29842918acc045d29ea0fa4adfff5b9c" is_pre_election: true
I20260924 00:21:48.055619 12608 raft_consensus.cc:2454] T 11ec8938396a435593baaca5e058677e P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08e4cf0195e645bcbb5ac5e3ede93096 in term 0.
I20260924 00:21:48.060845 13016 raft_consensus.cc:504] T 11ec8938396a435593baaca5e058677e P 08e4cf0195e645bcbb5ac5e3ede93096 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:48.063536 12755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11ec8938396a435593baaca5e058677e" candidate_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa329f7568e1416597742422ff0026cb"
I20260924 00:21:48.064078 12755 raft_consensus.cc:3037] T 11ec8938396a435593baaca5e058677e P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:48.070461 12755 raft_consensus.cc:2454] T 11ec8938396a435593baaca5e058677e P fa329f7568e1416597742422ff0026cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08e4cf0195e645bcbb5ac5e3ede93096 in term 1.
I20260924 00:21:48.071509 12832 leader_election.cc:302] T 11ec8938396a435593baaca5e058677e P 08e4cf0195e645bcbb5ac5e3ede93096 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08e4cf0195e645bcbb5ac5e3ede93096, fa329f7568e1416597742422ff0026cb; no voters:
I20260924 00:21:48.074419 13074 raft_consensus.cc:2781] T 11ec8938396a435593baaca5e058677e P 08e4cf0195e645bcbb5ac5e3ede93096 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:48.085074 13074 raft_consensus.cc:686] T 11ec8938396a435593baaca5e058677e P 08e4cf0195e645bcbb5ac5e3ede93096 [term 1 LEADER]: Becoming Leader. State: Replica: 08e4cf0195e645bcbb5ac5e3ede93096, State: Running, Role: LEADER
I20260924 00:21:48.087761 13074 consensus_queue.cc:237] T 11ec8938396a435593baaca5e058677e P 08e4cf0195e645bcbb5ac5e3ede93096 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:48.106491 12608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11ec8938396a435593baaca5e058677e" candidate_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29842918acc045d29ea0fa4adfff5b9c"
I20260924 00:21:48.107085 12608 raft_consensus.cc:3037] T 11ec8938396a435593baaca5e058677e P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:48.113168 12608 raft_consensus.cc:2454] T 11ec8938396a435593baaca5e058677e P 29842918acc045d29ea0fa4adfff5b9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08e4cf0195e645bcbb5ac5e3ede93096 in term 1.
I20260924 00:21:48.114563 13016 leader_election.cc:288] T 11ec8938396a435593baaca5e058677e P 08e4cf0195e645bcbb5ac5e3ede93096 [CANDIDATE]: Term 1 election: Requested vote from peers fa329f7568e1416597742422ff0026cb (127.10.4.66:40645), 29842918acc045d29ea0fa4adfff5b9c (127.10.4.65:46421)
W20260924 00:21:48.237943 12688 connection.cc:535] server connection from 127.0.0.1:38968 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260924 00:21:48.238880 12832 connection.cc:535] client connection to 127.10.4.125:46875 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260924 00:21:48.239033 12954 connection.cc:535] client connection to 127.10.4.125:46875 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260924 00:21:48.241214 12954 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.10.4.125:46875)
W20260924 00:21:48.242093 12954 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.10.4.125:46875)
W20260924 00:21:48.245146 12954 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.10.4.125:46875)
W20260924 00:21:48.246616 12954 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.10.4.125:46875)
W20260924 00:21:48.250522 12688 connection.cc:535] client connection to 127.10.4.125:46875 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260924 00:21:48.252131 12541 connection.cc:535] client connection to 127.10.4.125:46875 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260924 00:21:48.254379 12834 connection.cc:535] server connection from 127.0.0.1:36822 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260924 00:21:48.255188 12237 connection.cc:535] server connection from 127.0.0.1:39966 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260924 00:21:48.257011 13073 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:21:48.243304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.4.125:46875 (local address 127.10.4.67:35483)
0924 00:21:48.244032 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0924 00:21:48.244075 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 00:21:48.244303 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:21:48.256028 (+ 11725us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":307}
W20260924 00:21:48.259325 13057 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:21:48.253627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.4.125:46875 (local address 127.10.4.65:54513)
0924 00:21:48.254101 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0924 00:21:48.254141 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 00:21:48.254352 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:21:48.258307 (+ 3955us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":63}
W20260924 00:21:48.259333 13014 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:21:48.251946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.4.125:46875 (local address 127.10.4.66:54327)
0924 00:21:48.252930 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0924 00:21:48.253003 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0924 00:21:48.253229 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:21:48.258305 (+ 5076us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":581}
W20260924 00:21:48.280332 13079 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:21:48.249051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.4.125:46875 (local address 127.0.0.1:33178)
0924 00:21:48.278341 (+ 29290us) negotiation.cc:107] Waiting for socket to connect
0924 00:21:48.278558 (+ 217us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":28844,"threads_started":1}
I20260924 00:21:48.294140 13067 raft_consensus.cc:482] T 71c636b9c7c24cfcaae4ca0596a45ae6 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:48.294548 13067 raft_consensus.cc:504] T 71c636b9c7c24cfcaae4ca0596a45ae6 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:48.296222 13067 leader_election.cc:288] T 71c636b9c7c24cfcaae4ca0596a45ae6 P fa329f7568e1416597742422ff0026cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29842918acc045d29ea0fa4adfff5b9c (127.10.4.65:46421), 08e4cf0195e645bcbb5ac5e3ede93096 (127.10.4.67:38139)
I20260924 00:21:48.297847 12608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71c636b9c7c24cfcaae4ca0596a45ae6" candidate_uuid: "fa329f7568e1416597742422ff0026cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29842918acc045d29ea0fa4adfff5b9c" is_pre_election: true
I20260924 00:21:48.297886 12898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71c636b9c7c24cfcaae4ca0596a45ae6" candidate_uuid: "fa329f7568e1416597742422ff0026cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" is_pre_election: true
I20260924 00:21:48.298480 12608 raft_consensus.cc:2454] T 71c636b9c7c24cfcaae4ca0596a45ae6 P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa329f7568e1416597742422ff0026cb in term 0.
I20260924 00:21:48.298478 12898 raft_consensus.cc:2454] T 71c636b9c7c24cfcaae4ca0596a45ae6 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa329f7568e1416597742422ff0026cb in term 0.
I20260924 00:21:48.299535 12691 leader_election.cc:302] T 71c636b9c7c24cfcaae4ca0596a45ae6 P fa329f7568e1416597742422ff0026cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29842918acc045d29ea0fa4adfff5b9c, fa329f7568e1416597742422ff0026cb; no voters:
I20260924 00:21:48.301357 13067 raft_consensus.cc:2781] T 71c636b9c7c24cfcaae4ca0596a45ae6 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:48.301657 13067 raft_consensus.cc:482] T 71c636b9c7c24cfcaae4ca0596a45ae6 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:48.301954 13067 raft_consensus.cc:3037] T 71c636b9c7c24cfcaae4ca0596a45ae6 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:48.308681 13067 raft_consensus.cc:504] T 71c636b9c7c24cfcaae4ca0596a45ae6 P fa329f7568e1416597742422ff0026cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:48.311472 12608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71c636b9c7c24cfcaae4ca0596a45ae6" candidate_uuid: "fa329f7568e1416597742422ff0026cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29842918acc045d29ea0fa4adfff5b9c"
I20260924 00:21:48.311990 12608 raft_consensus.cc:3037] T 71c636b9c7c24cfcaae4ca0596a45ae6 P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:48.312814 12898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71c636b9c7c24cfcaae4ca0596a45ae6" candidate_uuid: "fa329f7568e1416597742422ff0026cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08e4cf0195e645bcbb5ac5e3ede93096"
I20260924 00:21:48.313374 12898 raft_consensus.cc:3037] T 71c636b9c7c24cfcaae4ca0596a45ae6 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:48.318238 12608 raft_consensus.cc:2454] T 71c636b9c7c24cfcaae4ca0596a45ae6 P 29842918acc045d29ea0fa4adfff5b9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa329f7568e1416597742422ff0026cb in term 1.
I20260924 00:21:48.319096 12898 raft_consensus.cc:2454] T 71c636b9c7c24cfcaae4ca0596a45ae6 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa329f7568e1416597742422ff0026cb in term 1.
I20260924 00:21:48.319396 12691 leader_election.cc:302] T 71c636b9c7c24cfcaae4ca0596a45ae6 P fa329f7568e1416597742422ff0026cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29842918acc045d29ea0fa4adfff5b9c, fa329f7568e1416597742422ff0026cb; no voters:
I20260924 00:21:48.319976 13067 leader_election.cc:288] T 71c636b9c7c24cfcaae4ca0596a45ae6 P fa329f7568e1416597742422ff0026cb [CANDIDATE]: Term 1 election: Requested vote from peers 29842918acc045d29ea0fa4adfff5b9c (127.10.4.65:46421), 08e4cf0195e645bcbb5ac5e3ede93096 (127.10.4.67:38139)
I20260924 00:21:48.320396 13015 raft_consensus.cc:2781] T 71c636b9c7c24cfcaae4ca0596a45ae6 P fa329f7568e1416597742422ff0026cb [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:48.320808 13015 raft_consensus.cc:686] T 71c636b9c7c24cfcaae4ca0596a45ae6 P fa329f7568e1416597742422ff0026cb [term 1 LEADER]: Becoming Leader. State: Replica: fa329f7568e1416597742422ff0026cb, State: Running, Role: LEADER
I20260924 00:21:48.321281 13081 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fb563a88aa00432ba12db382b1435a39)
I20260924 00:21:48.321574 13015 consensus_queue.cc:237] T 71c636b9c7c24cfcaae4ca0596a45ae6 P fa329f7568e1416597742422ff0026cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:48.321710 13081 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:48.342798 13081 leader_election.cc:288] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875), dba2791dd9734eb4b84bbbf2f7d674b8 (127.10.4.124:45407)
W20260924 00:21:48.359321 12239 leader_election.cc:334] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer dba2791dd9734eb4b84bbbf2f7d674b8 (127.10.4.124:45407): Network error: Client connection negotiation failed: client connection to 127.10.4.124:45407: connect: Connection refused (error 111)
W20260924 00:21:48.362666 12236 leader_election.cc:334] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
I20260924 00:21:48.363008 12236 leader_election.cc:302] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3a72a9440740a4948a000656ccabdf; no voters: dba2791dd9734eb4b84bbbf2f7d674b8, fb563a88aa00432ba12db382b1435a39
I20260924 00:21:48.363566 13081 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 00:21:48.752442 13015 consensus_queue.cc:1035] T 1c3ffccda42840d88b039e39b9ecc2c7 P fa329f7568e1416597742422ff0026cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 00:21:48.783543 13067 consensus_queue.cc:1035] T 1c3ffccda42840d88b039e39b9ecc2c7 P fa329f7568e1416597742422ff0026cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:48.952083 13096 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fb563a88aa00432ba12db382b1435a39)
I20260924 00:21:48.952595 13096 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:48.954574 13096 leader_election.cc:288] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875), dba2791dd9734eb4b84bbbf2f7d674b8 (127.10.4.124:45407)
W20260924 00:21:48.974390 12236 leader_election.cc:334] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
W20260924 00:21:48.975265 12239 leader_election.cc:334] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer dba2791dd9734eb4b84bbbf2f7d674b8 (127.10.4.124:45407): Network error: Client connection negotiation failed: client connection to 127.10.4.124:45407: connect: Connection refused (error 111)
I20260924 00:21:48.975703 12239 leader_election.cc:302] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3a72a9440740a4948a000656ccabdf; no voters: dba2791dd9734eb4b84bbbf2f7d674b8, fb563a88aa00432ba12db382b1435a39
I20260924 00:21:48.976508 13096 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 00:21:49.128021 13086 consensus_queue.cc:1035] T 11ec8938396a435593baaca5e058677e P 08e4cf0195e645bcbb5ac5e3ede93096 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:49.169291 13091 consensus_queue.cc:1035] T 71c636b9c7c24cfcaae4ca0596a45ae6 P fa329f7568e1416597742422ff0026cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:49.177919 13101 consensus_queue.cc:1035] T 11ec8938396a435593baaca5e058677e P 08e4cf0195e645bcbb5ac5e3ede93096 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:49.194792 13091 consensus_queue.cc:1035] T 71c636b9c7c24cfcaae4ca0596a45ae6 P fa329f7568e1416597742422ff0026cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260924 00:21:49.085685 13044 debug-util.cc:398] Leaking SignalData structure 0x7b080001e820 after lost signal to thread 12990
W20260924 00:21:49.324836 12990 thread.cc:636] GCE (cloud detector) Time spent creating pthread: real 1.646s user 0.579s sys 0.785s
W20260924 00:21:49.325269 12990 thread.cc:602] GCE (cloud detector) Time spent starting thread: real 1.647s user 0.579s sys 0.785s
W20260924 00:21:47.727245 13047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:21:49.331229 13109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:21:49.334028 12990 server_base.cc:1034] running on GCE node
I20260924 00:21:49.335144 12990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:21:49.337567 12990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:21:49.338896 12990 hybrid_clock.cc:648] HybridClock initialized: now 1790209309338855 us; error 36 us; skew 500 ppm
I20260924 00:21:49.339787 12990 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 00:21:49.351752 12990 webserver.cc:460] Webserver started at http://127.10.4.124:36597/ using document root <none> and password file <none>
I20260924 00:21:49.352747 12990 fs_manager.cc:359] Metadata directory not provided
I20260924 00:21:49.353031 12990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:21:49.361274 12990 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.007s sys 0.000s
I20260924 00:21:49.365782 13114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:21:49.366880 12990 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260924 00:21:49.367290 12990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/data,/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/wal
uuid: "dba2791dd9734eb4b84bbbf2f7d674b8"
format_stamp: "Formatted at 2026-09-24 00:21:37 on dist-test-slave-g3nd"
I20260924 00:21:49.370240 12990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:21:49.415202 12990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:21:49.416996 12990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:21:49.417414 12990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:21:49.487030 12990 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.124:45407
I20260924 00:21:49.487186 13165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.4.124:45407 every 8 connection(s)
I20260924 00:21:49.489856 12990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/data/info.pb
I20260924 00:21:49.491446 10257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu as pid 12990
I20260924 00:21:49.503458 13166 sys_catalog.cc:263] Verifying existing consensus state
I20260924 00:21:49.511348 13166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8: Bootstrap starting.
I20260924 00:21:49.527384 13166 log.cc:824] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8: Log is configured to *not* fsync() on all Append() calls
I20260924 00:21:49.542186 13166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 00:21:49.542968 13166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8: Bootstrap complete.
I20260924 00:21:49.562186 13166 raft_consensus.cc:348] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:49.564488 13166 raft_consensus.cc:729] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dba2791dd9734eb4b84bbbf2f7d674b8, State: Initialized, Role: FOLLOWER
I20260924 00:21:49.565333 13166 consensus_queue.cc:260] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:49.568269 13171 sys_catalog.cc:455] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } } }
I20260924 00:21:49.569139 13171 sys_catalog.cc:458] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:21:49.569778 13166 sys_catalog.cc:564] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:21:49.601090 13186 catalog_manager.cc:1259] Loaded cluster ID: 575a504d93ea472a8cb263817ca44ba4
I20260924 00:21:49.601366 13186 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8: loading cluster ID for follower catalog manager: success
I20260924 00:21:49.608631 13186 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8: acquiring CA information for follower catalog manager: success
I20260924 00:21:49.618422 13186 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 00:21:49.634557 12990 master_runner.cc:186] Error getting master registration for 127.10.4.125:46875: Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111),
I20260924 00:21:49.665088 12990 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:21:49.666252 13190 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fb563a88aa00432ba12db382b1435a39)
I20260924 00:21:49.666684 13190 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:49.668515 13190 leader_election.cc:288] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875), dba2791dd9734eb4b84bbbf2f7d674b8 (127.10.4.124:45407)
I20260924 00:21:49.707001 13141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3c3a72a9440740a4948a000656ccabdf" candidate_term: 3 candidate_status { last_received { term: 2 index: 22 } } ignore_live_leader: false dest_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" is_pre_election: true
I20260924 00:21:49.708058 13141 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3a72a9440740a4948a000656ccabdf in term 1.
W20260924 00:21:49.710811 12236 leader_election.cc:334] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
I20260924 00:21:49.711350 12239 leader_election.cc:302] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3a72a9440740a4948a000656ccabdf, dba2791dd9734eb4b84bbbf2f7d674b8; no voters: fb563a88aa00432ba12db382b1435a39
I20260924 00:21:49.713068 13190 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 00:21:49.713382 13190 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 2 FOLLOWER]: Starting leader election (detected failure of leader fb563a88aa00432ba12db382b1435a39)
I20260924 00:21:49.713681 13190 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 2 FOLLOWER]: Advancing to term 3
I20260924 00:21:49.722431 13190 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:49.734073 13190 leader_election.cc:288] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [CANDIDATE]: Term 3 election: Requested vote from peers fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875), dba2791dd9734eb4b84bbbf2f7d674b8 (127.10.4.124:45407)
I20260924 00:21:49.734953 13141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3c3a72a9440740a4948a000656ccabdf" candidate_term: 3 candidate_status { last_received { term: 2 index: 22 } } ignore_live_leader: false dest_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8"
I20260924 00:21:49.735483 13141 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 1 FOLLOWER]: Advancing to term 3
W20260924 00:21:49.740792 12236 leader_election.cc:334] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
I20260924 00:21:49.742434 12654 heartbeater.cc:347] Connected to a master server at 127.10.4.124:45407
I20260924 00:21:49.743017 12944 heartbeater.cc:347] Connected to a master server at 127.10.4.124:45407
I20260924 00:21:49.743544 12944 heartbeater.cc:503] Master 127.10.4.124:45407 was elected leader, sending a full tablet report...
I20260924 00:21:49.744511 13141 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3a72a9440740a4948a000656ccabdf in term 3.
I20260924 00:21:49.745543 12239 leader_election.cc:302] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3a72a9440740a4948a000656ccabdf, dba2791dd9734eb4b84bbbf2f7d674b8; no voters: fb563a88aa00432ba12db382b1435a39
I20260924 00:21:49.746289 13190 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 3 FOLLOWER]: Leader election won for term 3
I20260924 00:21:49.750730 13190 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [term 3 LEADER]: Becoming Leader. State: Replica: 3c3a72a9440740a4948a000656ccabdf, State: Running, Role: LEADER
I20260924 00:21:49.751893 13190 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 2.22, Last appended by leader: 22, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:49.753350 13130 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" instance_seqno: 1790209302017061) as {username='slave'} at 127.10.4.65:36789; Asking this server to re-register.
I20260924 00:21:49.754602 13131 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" instance_seqno: 1790209305592358) as {username='slave'} at 127.10.4.67:46645; Asking this server to re-register.
I20260924 00:21:49.754966 12654 heartbeater.cc:464] Registering TS with master...
I20260924 00:21:49.756142 12944 heartbeater.cc:464] Registering TS with master...
I20260924 00:21:49.762583 13129 ts_manager.cc:194] Registered new tserver with Master: 29842918acc045d29ea0fa4adfff5b9c (127.10.4.65:46421)
I20260924 00:21:49.765112 13196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c3a72a9440740a4948a000656ccabdf. Latest consensus state: current_term: 3 leader_uuid: "3c3a72a9440740a4948a000656ccabdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } } }
I20260924 00:21:49.765936 13196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [sys.catalog]: This master's current role is: LEADER
I20260924 00:21:49.766579 12800 heartbeater.cc:347] Connected to a master server at 127.10.4.124:45407
I20260924 00:21:49.767726 13130 ts_manager.cc:194] Registered new tserver with Master: 08e4cf0195e645bcbb5ac5e3ede93096 (127.10.4.67:38139)
I20260924 00:21:49.767792 13198 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:21:49.771688 13131 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "fa329f7568e1416597742422ff0026cb" instance_seqno: 1790209303799506) as {username='slave'} at 127.10.4.66:48045; Asking this server to re-register.
I20260924 00:21:49.773118 12800 heartbeater.cc:464] Registering TS with master...
I20260924 00:21:49.776849 13131 ts_manager.cc:194] Registered new tserver with Master: fa329f7568e1416597742422ff0026cb (127.10.4.66:40645)
I20260924 00:21:49.777455 13198 catalog_manager.cc:669] Loaded metadata for table default.table_7e0e41deeaa74586ad39386be2a8244d [id=0cf5ae7d6b56461fb17c887ec515c931]
I20260924 00:21:49.779098 13198 catalog_manager.cc:669] Loaded metadata for table default.table_027aad11c8d7454bbf92402722920c2a [id=291e8d50dc5248c280e4fde27a818707]
I20260924 00:21:49.780655 13198 catalog_manager.cc:669] Loaded metadata for table default.table_6888b4491c68473c8fbc8622820afc42 [id=670fab2a44eb41ee89d1a16ef0aef90c]
I20260924 00:21:49.787611 13198 tablet_loader.cc:96] loaded metadata for tablet 11ec8938396a435593baaca5e058677e (table default.table_6888b4491c68473c8fbc8622820afc42 [id=670fab2a44eb41ee89d1a16ef0aef90c])
I20260924 00:21:49.789494 13198 tablet_loader.cc:96] loaded metadata for tablet 1c3ffccda42840d88b039e39b9ecc2c7 (table default.table_7e0e41deeaa74586ad39386be2a8244d [id=0cf5ae7d6b56461fb17c887ec515c931])
I20260924 00:21:49.791815 13198 tablet_loader.cc:96] loaded metadata for tablet a237d2aeebfd4e1f8a0c2d76a6b2f9c7 (table default.table_027aad11c8d7454bbf92402722920c2a [id=291e8d50dc5248c280e4fde27a818707])
I20260924 00:21:49.792629 13198 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:21:49.795717 13198 catalog_manager.cc:1259] Loaded cluster ID: 575a504d93ea472a8cb263817ca44ba4
I20260924 00:21:49.795914 13198 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:21:49.799623 13198 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:21:49.802624 13198 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf: Loaded TSK: 0
I20260924 00:21:49.803791 13198 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:21:49.806697 13198 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260924 00:21:49.809717 13198 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260924 00:21:49.826954 13141 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Refusing update from remote peer 3c3a72a9440740a4948a000656ccabdf: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 24. (index mismatch)
I20260924 00:21:49.829128 13196 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [LEADER]: Connected to new peer: Peer: permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 4, Time since last communication: 0.000s
W20260924 00:21:49.830061 12236 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf -> Peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Couldn't send request to peer fb563a88aa00432ba12db382b1435a39. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 00:21:49.854036 12250 master_service.cc:946] ReplaceTablet: received request to replace tablet 71c636b9c7c24cfcaae4ca0596a45ae6 from {username='slave'} at 127.0.0.1:54444
I20260924 00:21:49.855019 12251 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54444:
table { table_name: "default.Table_6888b4491c68473c8fbc8622820afc42" } modify_external_catalogs: true
I20260924 00:21:49.855852 12251 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54444:
table { table_name: "default.Table_6888b4491c68473c8fbc8622820afc42" } modify_external_catalogs: true
I20260924 00:21:49.859637 12248 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54444:
name: "default.Table_b34b7e9f7270468b93a28c7329bbec21"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:49.859418 12249 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54444:
name: "default.Table_e720de17aede4a629b383fe3a2b473fe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:49.861176 12250 master_service.cc:946] ReplaceTablet: received request to replace tablet 71c636b9c7c24cfcaae4ca0596a45ae6 from {username='slave'} at 127.0.0.1:54444
I20260924 00:21:49.862743 12247 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:54444:
table { table_name: "default.Table_027aad11c8d7454bbf92402722920c2a" } new_table_name: "default.Table_d5742bff56654aa0b9199a9b1bb928ca" modify_external_catalogs: true
I20260924 00:21:49.865311 12250 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54444:
name: "default.Table_e43f7024e9b745758c0915ae7605e671"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:49.866259 12246 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54444:
name: "default.Table_4a8f7937d4ef462592a1c80eefab1d0d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:21:49.870900 12245 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:54444:
table { table_name: "default.Table_7e0e41deeaa74586ad39386be2a8244d" } new_table_name: "default.Table_f0b527f3d42148d69722944b46b07d72" modify_external_catalogs: true
F20260924 00:21:49.873380 12483 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_751162aa988a40a8a40c5f6c6c978860
*** Check failure stack trace: ***
@ 0x7fea783069a0 google::LogMessage::Flush() at ??:0
@ 0x7fea7830670f google::LogMessage::~LogMessage() at ??:0
@ 0x7fea833a1aea kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260924 00:21:49.938041 13171 sys_catalog.cc:455] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c3a72a9440740a4948a000656ccabdf. Latest consensus state: current_term: 3 leader_uuid: "3c3a72a9440740a4948a000656ccabdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } } }
I20260924 00:21:49.938745 13171 sys_catalog.cc:458] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:21:49.936422 13205 sys_catalog.cc:455] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "3c3a72a9440740a4948a000656ccabdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } } }
I20260924 00:21:49.939458 13205 sys_catalog.cc:458] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:21:49.945497 13205 sys_catalog.cc:455] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "3c3a72a9440740a4948a000656ccabdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } } }
I20260924 00:21:49.946158 13205 sys_catalog.cc:458] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:21:49.949998 13190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "3c3a72a9440740a4948a000656ccabdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } } }
I20260924 00:21:49.950727 13190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf [sys.catalog]: This master's current role is: LEADER
I20260924 00:21:49.952733 12252 catalog_manager.cc:5495] T 1c3ffccda42840d88b039e39b9ecc2c7 P fa329f7568e1416597742422ff0026cb reported cstate change: term changed from 0 to 1, leader changed from <none> to fa329f7568e1416597742422ff0026cb (127.10.4.66). New cstate: current_term: 1 leader_uuid: "fa329f7568e1416597742422ff0026cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } health_report { overall_health: HEALTHY } } }
@ 0x7fea833a133c kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7fea833a4fd2 kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7fea833a4f8a _ZNSt3__18__invokeIRZN4kudu6master30HmsNotificationLogListenerTask4InitEvE3$_0JEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOS6_DpOS7_ at ??:0
I20260924 00:21:50.016697 12587 tablet_service.cc:1467] Processing CreateTablet for tablet a237d2aeebfd4e1f8a0c2d76a6b2f9c7 (DEFAULT_TABLE table=default.table_027aad11c8d7454bbf92402722920c2a [id=291e8d50dc5248c280e4fde27a818707]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:50.018064 12587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a237d2aeebfd4e1f8a0c2d76a6b2f9c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:50.030622 12735 tablet_service.cc:1467] Processing CreateTablet for tablet a237d2aeebfd4e1f8a0c2d76a6b2f9c7 (DEFAULT_TABLE table=default.table_027aad11c8d7454bbf92402722920c2a [id=291e8d50dc5248c280e4fde27a818707]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:50.031701 12735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a237d2aeebfd4e1f8a0c2d76a6b2f9c7. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7fea833a4f1a std::__1::__invoke_void_return_wrapper<>::__call<>() at ??:0
I20260924 00:21:50.064167 13201 mvcc.cc:205] Tried to move back new op lower bound from 7332697333037932544 to 7332697317982625792. Current Snapshot: MvccSnapshot[applied={T|T < 7332697333037932544 or (T in {7332697333037932544})}]
@ 0x7fea833a4ee2 std::__1::__function::__alloc_func<>::operator()() at ??:0
I20260924 00:21:50.075976 12879 tablet_service.cc:1467] Processing CreateTablet for tablet a237d2aeebfd4e1f8a0c2d76a6b2f9c7 (DEFAULT_TABLE table=default.table_027aad11c8d7454bbf92402722920c2a [id=291e8d50dc5248c280e4fde27a818707]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:50.077807 12879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a237d2aeebfd4e1f8a0c2d76a6b2f9c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:50.079250 12244 master_service.cc:946] ReplaceTablet: received request to replace tablet 11ec8938396a435593baaca5e058677e from {username='slave'} at 127.0.0.1:54444
I20260924 00:21:50.079948 12252 master_service.cc:946] ReplaceTablet: received request to replace tablet 11ec8938396a435593baaca5e058677e from {username='slave'} at 127.0.0.1:54444
I20260924 00:21:50.078733 13221 tablet_bootstrap.cc:492] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P fa329f7568e1416597742422ff0026cb: Bootstrap starting.
I20260924 00:21:50.090485 13222 tablet_bootstrap.cc:492] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P 29842918acc045d29ea0fa4adfff5b9c: Bootstrap starting.
I20260924 00:21:50.096732 12244 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3c3a72a9440740a4948a000656ccabdf: Sending DeleteTablet for 3 replicas of tablet 11ec8938396a435593baaca5e058677e
I20260924 00:21:50.098465 12244 catalog_manager.cc:6565] ReplaceTablet: tablet 11ec8938396a435593baaca5e058677e deleted and replaced by tablet c45f49ad9c754827920cd966eebc693a
I20260924 00:21:50.101231 12735 tablet_service.cc:1514] Processing DeleteTablet for tablet 11ec8938396a435593baaca5e058677e with delete_type TABLET_DATA_DELETED (replaced by tablet c45f49ad9c754827920cd966eebc693a) from {username='slave'} at 127.0.0.1:38972
I20260924 00:21:50.104667 12878 tablet_service.cc:1514] Processing DeleteTablet for tablet 11ec8938396a435593baaca5e058677e with delete_type TABLET_DATA_DELETED (replaced by tablet c45f49ad9c754827920cd966eebc693a) from {username='slave'} at 127.0.0.1:36838
I20260924 00:21:50.105993 13221 tablet_bootstrap.cc:654] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P fa329f7568e1416597742422ff0026cb: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:50.107982 12587 tablet_service.cc:1514] Processing DeleteTablet for tablet 11ec8938396a435593baaca5e058677e with delete_type TABLET_DATA_DELETED (replaced by tablet c45f49ad9c754827920cd966eebc693a) from {username='slave'} at 127.0.0.1:33380
I20260924 00:21:50.117498 13226 tablet_replica.cc:330] T 11ec8938396a435593baaca5e058677e P fa329f7568e1416597742422ff0026cb: stopping tablet replica
I20260924 00:21:50.118212 13226 raft_consensus.cc:2229] T 11ec8938396a435593baaca5e058677e P fa329f7568e1416597742422ff0026cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:50.118666 13226 raft_consensus.cc:2258] T 11ec8938396a435593baaca5e058677e P fa329f7568e1416597742422ff0026cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:50.120851 13226 ts_tablet_manager.cc:1905] T 11ec8938396a435593baaca5e058677e P fa329f7568e1416597742422ff0026cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:21:50.122617 13227 tablet_replica.cc:330] T 11ec8938396a435593baaca5e058677e P 08e4cf0195e645bcbb5ac5e3ede93096: stopping tablet replica
I20260924 00:21:50.123343 13227 raft_consensus.cc:2229] T 11ec8938396a435593baaca5e058677e P 08e4cf0195e645bcbb5ac5e3ede93096 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:21:50.124317 13227 raft_consensus.cc:2258] T 11ec8938396a435593baaca5e058677e P 08e4cf0195e645bcbb5ac5e3ede93096 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:50.127449 13227 ts_tablet_manager.cc:1905] T 11ec8938396a435593baaca5e058677e P 08e4cf0195e645bcbb5ac5e3ede93096: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7fea833a41de std::__1::__function::__func<>::operator()() at ??:0
I20260924 00:21:50.132661 12588 tablet_service.cc:1467] Processing CreateTablet for tablet c45f49ad9c754827920cd966eebc693a (DEFAULT_TABLE table=default.table_6888b4491c68473c8fbc8622820afc42 [id=670fab2a44eb41ee89d1a16ef0aef90c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:50.133812 12588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c45f49ad9c754827920cd966eebc693a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:50.133846 13224 tablet_bootstrap.cc:492] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P 08e4cf0195e645bcbb5ac5e3ede93096: Bootstrap starting.
I20260924 00:21:50.134255 12735 tablet_service.cc:1467] Processing CreateTablet for tablet c45f49ad9c754827920cd966eebc693a (DEFAULT_TABLE table=default.table_6888b4491c68473c8fbc8622820afc42 [id=670fab2a44eb41ee89d1a16ef0aef90c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:50.135305 12735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c45f49ad9c754827920cd966eebc693a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:50.135488 13222 tablet_bootstrap.cc:654] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P 29842918acc045d29ea0fa4adfff5b9c: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:50.137943 12878 tablet_service.cc:1467] Processing CreateTablet for tablet c45f49ad9c754827920cd966eebc693a (DEFAULT_TABLE table=default.table_6888b4491c68473c8fbc8622820afc42 [id=670fab2a44eb41ee89d1a16ef0aef90c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:21:50.140501 12802 heartbeater.cc:503] Master 127.10.4.126:37913 was elected leader, sending a full tablet report...
I20260924 00:21:50.140903 12878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c45f49ad9c754827920cd966eebc693a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:21:50.145484 13221 tablet_bootstrap.cc:492] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P fa329f7568e1416597742422ff0026cb: No bootstrap required, opened a new log
I20260924 00:21:50.145876 13221 ts_tablet_manager.cc:1397] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P fa329f7568e1416597742422ff0026cb: Time spent bootstrapping tablet: real 0.067s user 0.010s sys 0.000s
I20260924 00:21:50.144706 13224 tablet_bootstrap.cc:654] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P 08e4cf0195e645bcbb5ac5e3ede93096: Neither blocks nor log segments found. Creating new log.
W20260924 00:21:50.148298 12252 catalog_manager.cc:5328] Ignoring report from unknown tablet 71c636b9c7c24cfcaae4ca0596a45ae6
I20260924 00:21:50.148128 13221 raft_consensus.cc:348] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:50.148782 13221 raft_consensus.cc:374] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:50.149081 13221 raft_consensus.cc:729] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa329f7568e1416597742422ff0026cb, State: Initialized, Role: FOLLOWER
I20260924 00:21:50.143599 13228 tablet_replica.cc:330] T 11ec8938396a435593baaca5e058677e P 29842918acc045d29ea0fa4adfff5b9c: stopping tablet replica
I20260924 00:21:50.150230 13228 raft_consensus.cc:2229] T 11ec8938396a435593baaca5e058677e P 29842918acc045d29ea0fa4adfff5b9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:21:50.150887 13228 raft_consensus.cc:2258] T 11ec8938396a435593baaca5e058677e P 29842918acc045d29ea0fa4adfff5b9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:21:50.153796 13228 ts_tablet_manager.cc:1905] T 11ec8938396a435593baaca5e058677e P 29842918acc045d29ea0fa4adfff5b9c: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x580785 std::__1::__function::__value_func<>::operator()() at ??:?
@ 0x5805ba std::__1::function<>::operator()() at ??:?
I20260924 00:21:50.165247 13221 consensus_queue.cc:260] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P fa329f7568e1416597742422ff0026cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:50.173508 13221 ts_tablet_manager.cc:1428] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P fa329f7568e1416597742422ff0026cb: Time spent starting tablet: real 0.027s user 0.006s sys 0.000s
I20260924 00:21:50.178484 13224 tablet_bootstrap.cc:492] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P 08e4cf0195e645bcbb5ac5e3ede93096: No bootstrap required, opened a new log
I20260924 00:21:50.178886 13224 ts_tablet_manager.cc:1397] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P 08e4cf0195e645bcbb5ac5e3ede93096: Time spent bootstrapping tablet: real 0.045s user 0.000s sys 0.009s
I20260924 00:21:50.182022 13224 raft_consensus.cc:348] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:50.182701 13224 raft_consensus.cc:374] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:50.182955 13224 raft_consensus.cc:729] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08e4cf0195e645bcbb5ac5e3ede93096, State: Initialized, Role: FOLLOWER
I20260924 00:21:50.185220 13224 consensus_queue.cc:260] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P 08e4cf0195e645bcbb5ac5e3ede93096 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:50.190990 13222 tablet_bootstrap.cc:492] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P 29842918acc045d29ea0fa4adfff5b9c: No bootstrap required, opened a new log
I20260924 00:21:50.191408 13222 ts_tablet_manager.cc:1397] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P 29842918acc045d29ea0fa4adfff5b9c: Time spent bootstrapping tablet: real 0.101s user 0.020s sys 0.000s
I20260924 00:21:50.193974 13222 raft_consensus.cc:348] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:50.194716 13222 raft_consensus.cc:374] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:50.194981 13222 raft_consensus.cc:729] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29842918acc045d29ea0fa4adfff5b9c, State: Initialized, Role: FOLLOWER
I20260924 00:21:50.196658 13224 ts_tablet_manager.cc:1428] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P 08e4cf0195e645bcbb5ac5e3ede93096: Time spent starting tablet: real 0.018s user 0.005s sys 0.002s
I20260924 00:21:50.200078 13226 ts_tablet_manager.cc:1918] T 11ec8938396a435593baaca5e058677e P fa329f7568e1416597742422ff0026cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:50.200454 13226 log.cc:1196] T 11ec8938396a435593baaca5e058677e P fa329f7568e1416597742422ff0026cb: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-1/wal/wals/11ec8938396a435593baaca5e058677e
I20260924 00:21:50.201223 13226 ts_tablet_manager.cc:1939] T 11ec8938396a435593baaca5e058677e P fa329f7568e1416597742422ff0026cb: Deleting consensus metadata
I20260924 00:21:50.201404 13221 tablet_bootstrap.cc:492] T c45f49ad9c754827920cd966eebc693a P fa329f7568e1416597742422ff0026cb: Bootstrap starting.
I20260924 00:21:50.203787 12236 catalog_manager.cc:4841] TS fa329f7568e1416597742422ff0026cb (127.10.4.66:40645): tablet 11ec8938396a435593baaca5e058677e (table default.table_6888b4491c68473c8fbc8622820afc42 [id=670fab2a44eb41ee89d1a16ef0aef90c]) successfully deleted
I20260924 00:21:50.205258 13222 consensus_queue.cc:260] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P 29842918acc045d29ea0fa4adfff5b9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:50.208033 12948 heartbeater.cc:503] Master 127.10.4.126:37913 was elected leader, sending a full tablet report...
I20260924 00:21:50.209991 13221 tablet_bootstrap.cc:654] T c45f49ad9c754827920cd966eebc693a P fa329f7568e1416597742422ff0026cb: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:50.211275 13228 ts_tablet_manager.cc:1918] T 11ec8938396a435593baaca5e058677e P 29842918acc045d29ea0fa4adfff5b9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:50.211678 13228 log.cc:1196] T 11ec8938396a435593baaca5e058677e P 29842918acc045d29ea0fa4adfff5b9c: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-0/wal/wals/11ec8938396a435593baaca5e058677e
I20260924 00:21:50.212473 13228 ts_tablet_manager.cc:1939] T 11ec8938396a435593baaca5e058677e P 29842918acc045d29ea0fa4adfff5b9c: Deleting consensus metadata
W20260924 00:21:50.212610 12252 catalog_manager.cc:5328] Ignoring report from unknown tablet 71c636b9c7c24cfcaae4ca0596a45ae6
I20260924 00:21:50.215724 12239 catalog_manager.cc:4841] TS 29842918acc045d29ea0fa4adfff5b9c (127.10.4.65:46421): tablet 11ec8938396a435593baaca5e058677e (table default.table_6888b4491c68473c8fbc8622820afc42 [id=670fab2a44eb41ee89d1a16ef0aef90c]) successfully deleted
I20260924 00:21:50.219520 13221 tablet_bootstrap.cc:492] T c45f49ad9c754827920cd966eebc693a P fa329f7568e1416597742422ff0026cb: No bootstrap required, opened a new log
I20260924 00:21:50.219982 13221 ts_tablet_manager.cc:1397] T c45f49ad9c754827920cd966eebc693a P fa329f7568e1416597742422ff0026cb: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.003s
I20260924 00:21:50.220665 13224 tablet_bootstrap.cc:492] T c45f49ad9c754827920cd966eebc693a P 08e4cf0195e645bcbb5ac5e3ede93096: Bootstrap starting.
I20260924 00:21:50.222769 13221 raft_consensus.cc:348] T c45f49ad9c754827920cd966eebc693a P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
@ 0x7fea7a4b7dc7 kudu::Thread::SuperviseThread() at ??:0
@ 0x4230f0 __tsan_thread_start_func at ??:?
I20260924 00:21:50.224643 13221 raft_consensus.cc:374] T c45f49ad9c754827920cd966eebc693a P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:50.224917 13221 raft_consensus.cc:729] T c45f49ad9c754827920cd966eebc693a P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa329f7568e1416597742422ff0026cb, State: Initialized, Role: FOLLOWER
I20260924 00:21:50.227222 13222 ts_tablet_manager.cc:1428] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P 29842918acc045d29ea0fa4adfff5b9c: Time spent starting tablet: real 0.036s user 0.006s sys 0.002s
I20260924 00:21:50.229755 13222 tablet_bootstrap.cc:492] T c45f49ad9c754827920cd966eebc693a P 29842918acc045d29ea0fa4adfff5b9c: Bootstrap starting.
I20260924 00:21:50.235395 13224 tablet_bootstrap.cc:654] T c45f49ad9c754827920cd966eebc693a P 08e4cf0195e645bcbb5ac5e3ede93096: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:50.236850 13222 tablet_bootstrap.cc:654] T c45f49ad9c754827920cd966eebc693a P 29842918acc045d29ea0fa4adfff5b9c: Neither blocks nor log segments found. Creating new log.
I20260924 00:21:50.243372 12656 heartbeater.cc:503] Master 127.10.4.126:37913 was elected leader, sending a full tablet report...
I20260924 00:21:50.243443 13221 consensus_queue.cc:260] T c45f49ad9c754827920cd966eebc693a P fa329f7568e1416597742422ff0026cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
W20260924 00:21:50.250557 12252 catalog_manager.cc:5328] Ignoring report from unknown tablet 71c636b9c7c24cfcaae4ca0596a45ae6
I20260924 00:21:50.255656 13221 ts_tablet_manager.cc:1428] T c45f49ad9c754827920cd966eebc693a P fa329f7568e1416597742422ff0026cb: Time spent starting tablet: real 0.035s user 0.004s sys 0.002s
I20260924 00:21:50.257833 13227 ts_tablet_manager.cc:1918] T 11ec8938396a435593baaca5e058677e P 08e4cf0195e645bcbb5ac5e3ede93096: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 00:21:50.258327 13227 log.cc:1196] T 11ec8938396a435593baaca5e058677e P 08e4cf0195e645bcbb5ac5e3ede93096: Deleting WAL directory at /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/ts-2/wal/wals/11ec8938396a435593baaca5e058677e
I20260924 00:21:50.265215 13202 raft_consensus.cc:482] T c45f49ad9c754827920cd966eebc693a P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
@ 0x7fea7bb256db start_thread at ??:0
I20260924 00:21:50.265650 13202 raft_consensus.cc:504] T c45f49ad9c754827920cd966eebc693a P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:50.267616 13202 leader_election.cc:288] T c45f49ad9c754827920cd966eebc693a P fa329f7568e1416597742422ff0026cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29842918acc045d29ea0fa4adfff5b9c (127.10.4.65:46421), 08e4cf0195e645bcbb5ac5e3ede93096 (127.10.4.67:38139)
I20260924 00:21:50.270483 13227 ts_tablet_manager.cc:1939] T 11ec8938396a435593baaca5e058677e P 08e4cf0195e645bcbb5ac5e3ede93096: Deleting consensus metadata
I20260924 00:21:50.272710 12898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c45f49ad9c754827920cd966eebc693a" candidate_uuid: "fa329f7568e1416597742422ff0026cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" is_pre_election: true
I20260924 00:21:50.273150 12239 catalog_manager.cc:4841] TS 08e4cf0195e645bcbb5ac5e3ede93096 (127.10.4.67:38139): tablet 11ec8938396a435593baaca5e058677e (table default.table_6888b4491c68473c8fbc8622820afc42 [id=670fab2a44eb41ee89d1a16ef0aef90c]) successfully deleted
I20260924 00:21:50.274535 12608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c45f49ad9c754827920cd966eebc693a" candidate_uuid: "fa329f7568e1416597742422ff0026cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29842918acc045d29ea0fa4adfff5b9c" is_pre_election: true
W20260924 00:21:50.274884 12691 leader_election.cc:341] T c45f49ad9c754827920cd966eebc693a P fa329f7568e1416597742422ff0026cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 08e4cf0195e645bcbb5ac5e3ede93096 (127.10.4.67:38139): Illegal state: must be running to vote when last-logged opid is not known
W20260924 00:21:50.276041 12691 leader_election.cc:341] T c45f49ad9c754827920cd966eebc693a P fa329f7568e1416597742422ff0026cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29842918acc045d29ea0fa4adfff5b9c (127.10.4.65:46421): Illegal state: must be running to vote when last-logged opid is not known
I20260924 00:21:50.276440 13224 tablet_bootstrap.cc:492] T c45f49ad9c754827920cd966eebc693a P 08e4cf0195e645bcbb5ac5e3ede93096: No bootstrap required, opened a new log
I20260924 00:21:50.276413 12691 leader_election.cc:302] T c45f49ad9c754827920cd966eebc693a P fa329f7568e1416597742422ff0026cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa329f7568e1416597742422ff0026cb; no voters: 08e4cf0195e645bcbb5ac5e3ede93096, 29842918acc045d29ea0fa4adfff5b9c
I20260924 00:21:50.276841 13224 ts_tablet_manager.cc:1397] T c45f49ad9c754827920cd966eebc693a P 08e4cf0195e645bcbb5ac5e3ede93096: Time spent bootstrapping tablet: real 0.056s user 0.014s sys 0.003s
I20260924 00:21:50.277449 13202 raft_consensus.cc:2726] T c45f49ad9c754827920cd966eebc693a P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:21:50.279439 13222 tablet_bootstrap.cc:492] T c45f49ad9c754827920cd966eebc693a P 29842918acc045d29ea0fa4adfff5b9c: No bootstrap required, opened a new log
I20260924 00:21:50.279268 13224 raft_consensus.cc:348] T c45f49ad9c754827920cd966eebc693a P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:50.279848 13222 ts_tablet_manager.cc:1397] T c45f49ad9c754827920cd966eebc693a P 29842918acc045d29ea0fa4adfff5b9c: Time spent bootstrapping tablet: real 0.050s user 0.003s sys 0.008s
@ 0x7fea76fab61f clone at ??:0
I20260924 00:21:50.279958 13224 raft_consensus.cc:374] T c45f49ad9c754827920cd966eebc693a P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:50.280892 13224 raft_consensus.cc:729] T c45f49ad9c754827920cd966eebc693a P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08e4cf0195e645bcbb5ac5e3ede93096, State: Initialized, Role: FOLLOWER
I20260924 00:21:50.281630 13224 consensus_queue.cc:260] T c45f49ad9c754827920cd966eebc693a P 08e4cf0195e645bcbb5ac5e3ede93096 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:50.282609 13222 raft_consensus.cc:348] T c45f49ad9c754827920cd966eebc693a P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:50.283298 13222 raft_consensus.cc:374] T c45f49ad9c754827920cd966eebc693a P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:21:50.283576 13222 raft_consensus.cc:729] T c45f49ad9c754827920cd966eebc693a P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29842918acc045d29ea0fa4adfff5b9c, State: Initialized, Role: FOLLOWER
I20260924 00:21:50.284289 13222 consensus_queue.cc:260] T c45f49ad9c754827920cd966eebc693a P 29842918acc045d29ea0fa4adfff5b9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:50.288568 13222 ts_tablet_manager.cc:1428] T c45f49ad9c754827920cd966eebc693a P 29842918acc045d29ea0fa4adfff5b9c: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260924 00:21:50.289187 13224 ts_tablet_manager.cc:1428] T c45f49ad9c754827920cd966eebc693a P 08e4cf0195e645bcbb5ac5e3ede93096: Time spent starting tablet: real 0.012s user 0.005s sys 0.001s
I20260924 00:21:50.434406 13212 raft_consensus.cc:482] T c45f49ad9c754827920cd966eebc693a P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:50.434890 13212 raft_consensus.cc:504] T c45f49ad9c754827920cd966eebc693a P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:50.436777 13212 leader_election.cc:288] T c45f49ad9c754827920cd966eebc693a P 08e4cf0195e645bcbb5ac5e3ede93096 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29842918acc045d29ea0fa4adfff5b9c (127.10.4.65:46421), fa329f7568e1416597742422ff0026cb (127.10.4.66:40645)
I20260924 00:21:50.437683 12608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c45f49ad9c754827920cd966eebc693a" candidate_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29842918acc045d29ea0fa4adfff5b9c" is_pre_election: true
I20260924 00:21:50.438310 12608 raft_consensus.cc:2454] T c45f49ad9c754827920cd966eebc693a P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08e4cf0195e645bcbb5ac5e3ede93096 in term 0.
I20260924 00:21:50.439348 12755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c45f49ad9c754827920cd966eebc693a" candidate_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa329f7568e1416597742422ff0026cb" is_pre_election: true
I20260924 00:21:50.439800 12835 leader_election.cc:302] T c45f49ad9c754827920cd966eebc693a P 08e4cf0195e645bcbb5ac5e3ede93096 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08e4cf0195e645bcbb5ac5e3ede93096, 29842918acc045d29ea0fa4adfff5b9c; no voters:
I20260924 00:21:50.439977 12755 raft_consensus.cc:2454] T c45f49ad9c754827920cd966eebc693a P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08e4cf0195e645bcbb5ac5e3ede93096 in term 0.
I20260924 00:21:50.440830 13212 raft_consensus.cc:2781] T c45f49ad9c754827920cd966eebc693a P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:50.441149 13212 raft_consensus.cc:482] T c45f49ad9c754827920cd966eebc693a P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:50.441398 13212 raft_consensus.cc:3037] T c45f49ad9c754827920cd966eebc693a P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:50.447233 13212 raft_consensus.cc:504] T c45f49ad9c754827920cd966eebc693a P 08e4cf0195e645bcbb5ac5e3ede93096 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:50.449325 13212 leader_election.cc:288] T c45f49ad9c754827920cd966eebc693a P 08e4cf0195e645bcbb5ac5e3ede93096 [CANDIDATE]: Term 1 election: Requested vote from peers 29842918acc045d29ea0fa4adfff5b9c (127.10.4.65:46421), fa329f7568e1416597742422ff0026cb (127.10.4.66:40645)
I20260924 00:21:50.450170 12608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c45f49ad9c754827920cd966eebc693a" candidate_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29842918acc045d29ea0fa4adfff5b9c"
I20260924 00:21:50.450511 12755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c45f49ad9c754827920cd966eebc693a" candidate_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa329f7568e1416597742422ff0026cb"
I20260924 00:21:50.450685 12608 raft_consensus.cc:3037] T c45f49ad9c754827920cd966eebc693a P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:50.451061 12755 raft_consensus.cc:3037] T c45f49ad9c754827920cd966eebc693a P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:50.456383 12608 raft_consensus.cc:2454] T c45f49ad9c754827920cd966eebc693a P 29842918acc045d29ea0fa4adfff5b9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08e4cf0195e645bcbb5ac5e3ede93096 in term 1.
I20260924 00:21:50.457137 12755 raft_consensus.cc:2454] T c45f49ad9c754827920cd966eebc693a P fa329f7568e1416597742422ff0026cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08e4cf0195e645bcbb5ac5e3ede93096 in term 1.
I20260924 00:21:50.457427 12835 leader_election.cc:302] T c45f49ad9c754827920cd966eebc693a P 08e4cf0195e645bcbb5ac5e3ede93096 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08e4cf0195e645bcbb5ac5e3ede93096, 29842918acc045d29ea0fa4adfff5b9c; no voters:
I20260924 00:21:50.458196 13212 raft_consensus.cc:2781] T c45f49ad9c754827920cd966eebc693a P 08e4cf0195e645bcbb5ac5e3ede93096 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:50.458598 13212 raft_consensus.cc:686] T c45f49ad9c754827920cd966eebc693a P 08e4cf0195e645bcbb5ac5e3ede93096 [term 1 LEADER]: Becoming Leader. State: Replica: 08e4cf0195e645bcbb5ac5e3ede93096, State: Running, Role: LEADER
I20260924 00:21:50.459349 13212 consensus_queue.cc:237] T c45f49ad9c754827920cd966eebc693a P 08e4cf0195e645bcbb5ac5e3ede93096 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
W20260924 00:21:50.481360 12957 connection.cc:535] client connection to 127.10.4.126:37913 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260924 00:21:50.481568 12835 connection.cc:535] client connection to 127.10.4.126:37913 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260924 00:21:50.481774 12544 connection.cc:535] client connection to 127.10.4.126:37913 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260924 00:21:50.483013 12957 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.10.4.126:37913)
W20260924 00:21:50.483714 12957 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.10.4.126:37913)
W20260924 00:21:50.484346 12957 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.10.4.126:37913)
W20260924 00:21:50.487044 12957 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.10.4.126:37913)
I20260924 00:21:50.598711 13212 raft_consensus.cc:482] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:50.599109 13212 raft_consensus.cc:504] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:50.600778 13212 leader_election.cc:288] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P 08e4cf0195e645bcbb5ac5e3ede93096 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa329f7568e1416597742422ff0026cb (127.10.4.66:40645), 29842918acc045d29ea0fa4adfff5b9c (127.10.4.65:46421)
I20260924 00:21:50.601706 12755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a237d2aeebfd4e1f8a0c2d76a6b2f9c7" candidate_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa329f7568e1416597742422ff0026cb" is_pre_election: true
I20260924 00:21:50.601878 12608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a237d2aeebfd4e1f8a0c2d76a6b2f9c7" candidate_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29842918acc045d29ea0fa4adfff5b9c" is_pre_election: true
I20260924 00:21:50.602195 12755 raft_consensus.cc:2454] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08e4cf0195e645bcbb5ac5e3ede93096 in term 0.
I20260924 00:21:50.602516 12608 raft_consensus.cc:2454] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08e4cf0195e645bcbb5ac5e3ede93096 in term 0.
I20260924 00:21:50.603063 12832 leader_election.cc:302] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P 08e4cf0195e645bcbb5ac5e3ede93096 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08e4cf0195e645bcbb5ac5e3ede93096, fa329f7568e1416597742422ff0026cb; no voters:
I20260924 00:21:50.603826 13212 raft_consensus.cc:2781] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:21:50.604173 13212 raft_consensus.cc:482] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:21:50.604578 13212 raft_consensus.cc:3037] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:50.611029 13212 raft_consensus.cc:504] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:50.613217 13212 leader_election.cc:288] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P 08e4cf0195e645bcbb5ac5e3ede93096 [CANDIDATE]: Term 1 election: Requested vote from peers fa329f7568e1416597742422ff0026cb (127.10.4.66:40645), 29842918acc045d29ea0fa4adfff5b9c (127.10.4.65:46421)
I20260924 00:21:50.613727 12755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a237d2aeebfd4e1f8a0c2d76a6b2f9c7" candidate_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa329f7568e1416597742422ff0026cb"
I20260924 00:21:50.613867 12608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a237d2aeebfd4e1f8a0c2d76a6b2f9c7" candidate_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29842918acc045d29ea0fa4adfff5b9c"
I20260924 00:21:50.614182 12755 raft_consensus.cc:3037] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P fa329f7568e1416597742422ff0026cb [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:50.614399 12608 raft_consensus.cc:3037] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P 29842918acc045d29ea0fa4adfff5b9c [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:21:50.618237 12755 raft_consensus.cc:2454] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P fa329f7568e1416597742422ff0026cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08e4cf0195e645bcbb5ac5e3ede93096 in term 1.
I20260924 00:21:50.619062 12832 leader_election.cc:302] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P 08e4cf0195e645bcbb5ac5e3ede93096 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08e4cf0195e645bcbb5ac5e3ede93096, fa329f7568e1416597742422ff0026cb; no voters:
I20260924 00:21:50.619668 13212 raft_consensus.cc:2781] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:21:50.620015 13212 raft_consensus.cc:686] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P 08e4cf0195e645bcbb5ac5e3ede93096 [term 1 LEADER]: Becoming Leader. State: Replica: 08e4cf0195e645bcbb5ac5e3ede93096, State: Running, Role: LEADER
I20260924 00:21:50.620280 12608 raft_consensus.cc:2454] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P 29842918acc045d29ea0fa4adfff5b9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08e4cf0195e645bcbb5ac5e3ede93096 in term 1.
I20260924 00:21:50.620769 13212 consensus_queue.cc:237] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P 08e4cf0195e645bcbb5ac5e3ede93096 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 } } peers { permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 } } peers { permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38139 } }
I20260924 00:21:50.631337 13245 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c3a72a9440740a4948a000656ccabdf)
I20260924 00:21:50.632798 13245 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
W20260924 00:21:50.651580 13118 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
I20260924 00:21:50.654209 13245 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:21:50.663916 13115 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
W20260924 00:21:50.669474 13118 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913): Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
I20260924 00:21:50.670027 13118 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dba2791dd9734eb4b84bbbf2f7d674b8; no voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39
I20260924 00:21:50.670872 13245 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 00:21:51.346222 13251 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c3a72a9440740a4948a000656ccabdf)
I20260924 00:21:51.346695 13251 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:51.348799 13251 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:21:51.371608 13115 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
W20260924 00:21:51.373644 13118 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913): Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
I20260924 00:21:51.374053 13118 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dba2791dd9734eb4b84bbbf2f7d674b8; no voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39
I20260924 00:21:51.374810 13251 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 00:21:51.439265 13255 consensus_queue.cc:1035] T c45f49ad9c754827920cd966eebc693a P 08e4cf0195e645bcbb5ac5e3ede93096 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:51.459306 13255 consensus_queue.cc:1035] T c45f49ad9c754827920cd966eebc693a P 08e4cf0195e645bcbb5ac5e3ede93096 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:51.648550 10257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu with pid 12990
W20260924 00:21:51.676779 12957 connection.cc:535] client connection to 127.10.4.124:45407 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260924 00:21:51.773231 13262 consensus_queue.cc:1035] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P 08e4cf0195e645bcbb5ac5e3ede93096 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:51.788215 13262 consensus_queue.cc:1035] T a237d2aeebfd4e1f8a0c2d76a6b2f9c7 P 08e4cf0195e645bcbb5ac5e3ede93096 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa329f7568e1416597742422ff0026cb" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:21:52.310293 10257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu
/tmp/dist-test-taskko7HPF/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.4.124:45407
--webserver_interface=127.10.4.124
--webserver_port=36597
--builtin_ntp_servers=127.10.4.84:44801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.4.126:37913,127.10.4.125:46875,127.10.4.124:45407
--hive_metastore_uris=thrift://127.0.0.1:45439
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260924 00:21:52.623612 13275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:21:52.624205 13275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:21:52.624653 13275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:21:52.657271 13275 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260924 00:21:52.657549 13275 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260924 00:21:52.657823 13275 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:21:52.658110 13275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:21:52.658358 13275 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:21:52.658596 13275 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:21:52.695482 13275 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.4.84:44801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:45439
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.10.4.126:37913,127.10.4.125:46875,127.10.4.124:45407
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.4.124:45407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.4.124
--webserver_port=36597
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:32:10 UTC on ccycloud.quasar-wfxeao.root.comops.site
build id 4668917
TSAN enabled
I20260924 00:21:52.696664 13275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:21:52.698222 13275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:21:52.716235 13287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:21:52.717362 13286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:21:52.716594 13289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:21:53.843165 13288 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection time-out
I20260924 00:21:53.843411 13275 server_base.cc:1029] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260924 00:21:53.847324 13275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:21:53.850189 13275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:21:53.851768 13275 hybrid_clock.cc:648] HybridClock initialized: now 1790209313851682 us; error 77 us; skew 500 ppm
I20260924 00:21:53.853056 13275 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 00:21:53.859892 13275 webserver.cc:460] Webserver started at http://127.10.4.124:36597/ using document root <none> and password file <none>
I20260924 00:21:53.860867 13275 fs_manager.cc:359] Metadata directory not provided
I20260924 00:21:53.861147 13275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:21:53.869588 13275 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.007s sys 0.000s
I20260924 00:21:53.874191 13305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:21:53.875495 13275 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260924 00:21:53.875921 13275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/data,/tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/wal
uuid: "dba2791dd9734eb4b84bbbf2f7d674b8"
format_stamp: "Formatted at 2026-09-24 00:21:37 on dist-test-slave-g3nd"
I20260924 00:21:53.878371 13275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:21:53.922287 13275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:21:53.923708 13275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:21:53.924117 13275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:21:53.994269 13275 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.124:45407
I20260924 00:21:53.994369 13357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.4.124:45407 every 8 connection(s)
I20260924 00:21:53.997280 13275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskko7HPF/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790209242852804-10257-0/minicluster-data/master-2/data/info.pb
I20260924 00:21:54.002883 10257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskko7HPF/build/tsan/bin/kudu as pid 13275
I20260924 00:21:54.009002 13358 sys_catalog.cc:263] Verifying existing consensus state
I20260924 00:21:54.014569 13358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8: Bootstrap starting.
I20260924 00:21:54.038475 13358 log.cc:824] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8: Log is configured to *not* fsync() on all Append() calls
I20260924 00:21:54.103124 13358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8: Bootstrap replayed 1/1 log segments. Stats: ops{read=28 overwritten=0 applied=28 ignored=0} inserts{seen=16 ignored=0} mutations{seen=24 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 00:21:54.103833 13358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8: Bootstrap complete.
I20260924 00:21:54.123389 13358 raft_consensus.cc:348] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:54.125535 13358 raft_consensus.cc:729] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: dba2791dd9734eb4b84bbbf2f7d674b8, State: Initialized, Role: FOLLOWER
I20260924 00:21:54.126436 13358 consensus_queue.cc:260] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 28, Last appended: 3.28, Last appended by leader: 28, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:54.129205 13363 sys_catalog.cc:455] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } } }
I20260924 00:21:54.129876 13363 sys_catalog.cc:458] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:21:54.130309 13358 sys_catalog.cc:564] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:21:54.161034 13377 catalog_manager.cc:1259] Loaded cluster ID: 575a504d93ea472a8cb263817ca44ba4
I20260924 00:21:54.161415 13377 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8: loading cluster ID for follower catalog manager: success
I20260924 00:21:54.175647 13377 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8: acquiring CA information for follower catalog manager: success
I20260924 00:21:54.181017 13377 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 00:21:54.157193 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:21:54.188920 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:21:54.318487 12654 heartbeater.cc:347] Connected to a master server at 127.10.4.124:45407
I20260924 00:21:54.326505 12800 heartbeater.cc:347] Connected to a master server at 127.10.4.124:45407
I20260924 00:21:54.326646 13323 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "29842918acc045d29ea0fa4adfff5b9c" instance_seqno: 1790209302017061) as {username='slave'} at 127.10.4.65:59515; Asking this server to re-register.
I20260924 00:21:54.328832 12654 heartbeater.cc:464] Registering TS with master...
I20260924 00:21:54.331544 13323 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "fa329f7568e1416597742422ff0026cb" instance_seqno: 1790209303799506) as {username='slave'} at 127.10.4.66:59687; Asking this server to re-register.
I20260924 00:21:54.333104 12800 heartbeater.cc:464] Registering TS with master...
I20260924 00:21:54.333398 13322 ts_manager.cc:194] Registered new tserver with Master: 29842918acc045d29ea0fa4adfff5b9c (127.10.4.65:46421)
I20260924 00:21:54.336674 13322 ts_manager.cc:194] Registered new tserver with Master: fa329f7568e1416597742422ff0026cb (127.10.4.66:40645)
I20260924 00:21:54.425408 12944 heartbeater.cc:347] Connected to a master server at 127.10.4.124:45407
I20260924 00:21:54.429507 13322 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "08e4cf0195e645bcbb5ac5e3ede93096" instance_seqno: 1790209305592358) as {username='slave'} at 127.10.4.67:53771; Asking this server to re-register.
I20260924 00:21:54.430717 12944 heartbeater.cc:464] Registering TS with master...
I20260924 00:21:54.434098 13322 ts_manager.cc:194] Registered new tserver with Master: 08e4cf0195e645bcbb5ac5e3ede93096 (127.10.4.67:38139)
I20260924 00:21:54.748364 13386 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:54.748832 13386 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:54.751156 13386 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:21:54.754781 13309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
W20260924 00:21:54.758526 13306 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
W20260924 00:21:54.759325 13309 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913): Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
I20260924 00:21:54.759704 13309 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dba2791dd9734eb4b84bbbf2f7d674b8; no voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39
I20260924 00:21:54.760378 13386 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 00:21:55.313757 13391 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:55.314129 13391 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:55.315605 13391 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:21:55.323693 13309 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913): Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
W20260924 00:21:55.325312 13306 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
I20260924 00:21:55.325701 13306 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dba2791dd9734eb4b84bbbf2f7d674b8; no voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39
I20260924 00:21:55.326370 13391 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 00:21:56.335224 10257 master-stress-test.cc:578] Killing test thread 1/10
I20260924 00:21:56.371551 13400 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:56.371937 13400 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:56.373344 13400 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:21:56.377712 13309 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913): Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
W20260924 00:21:56.378142 13306 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
I20260924 00:21:56.378433 13306 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dba2791dd9734eb4b84bbbf2f7d674b8; no voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39
I20260924 00:21:56.379033 13400 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 00:21:57.929080 13411 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:57.929466 13411 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:57.930954 13411 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:21:57.935280 13309 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913): Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
W20260924 00:21:57.936012 13306 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
I20260924 00:21:57.936357 13306 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dba2791dd9734eb4b84bbbf2f7d674b8; no voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39
I20260924 00:21:57.936883 13411 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 00:21:59.021190 13420 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:21:59.021595 13420 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:21:59.023167 13420 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:21:59.027347 13309 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913): Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
W20260924 00:21:59.027770 13306 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
I20260924 00:21:59.028124 13306 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dba2791dd9734eb4b84bbbf2f7d674b8; no voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39
I20260924 00:21:59.028640 13420 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 00:21:59.191038 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:21:59.191378 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:22:00.119856 13433 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:22:00.120247 13433 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:22:00.121958 13433 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:22:00.123939 13309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111) [suppressed 9 similar messages]
W20260924 00:22:00.127828 13309 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913): Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
W20260924 00:22:00.129304 13306 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
I20260924 00:22:00.129602 13306 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dba2791dd9734eb4b84bbbf2f7d674b8; no voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39
I20260924 00:22:00.130156 13433 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 00:22:04.194924 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:22:04.195343 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:22:04.355378 13497 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:22:04.355896 13497 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:22:04.357718 13497 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:22:04.361267 13309 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913): Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
W20260924 00:22:04.361749 13306 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
I20260924 00:22:04.362030 13306 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dba2791dd9734eb4b84bbbf2f7d674b8; no voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39
I20260924 00:22:04.362581 13497 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260924 00:22:05.024501 12796 debug-util.cc:398] Leaking SignalData structure 0x7b08000df7e0 after lost signal to thread 12665
W20260924 00:22:05.025501 12796 debug-util.cc:398] Leaking SignalData structure 0x7b08000db7a0 after lost signal to thread 12799
I20260924 00:22:09.198240 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:22:09.198594 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:22:12.883661 13612 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:22:12.884166 13612 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:22:12.886089 13612 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:22:12.888253 13309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260924 00:22:12.891439 13309 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913): Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
W20260924 00:22:12.892004 13306 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
I20260924 00:22:12.892310 13306 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dba2791dd9734eb4b84bbbf2f7d674b8; no voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39
I20260924 00:22:12.892871 13612 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 00:22:14.201689 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:22:14.202137 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:22:17.484035 13679 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:22:17.484411 13679 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:22:17.485970 13679 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:22:17.490209 13309 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913): Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
W20260924 00:22:17.490620 13306 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
I20260924 00:22:17.490931 13306 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dba2791dd9734eb4b84bbbf2f7d674b8; no voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39
I20260924 00:22:17.491437 13679 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 00:22:19.204757 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:22:19.205140 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260924 00:22:21.788823 13354 debug-util.cc:398] Leaking SignalData structure 0x7b080009c600 after lost signal to thread 13280
W20260924 00:22:21.789409 13354 debug-util.cc:398] Leaking SignalData structure 0x7b08000a0b40 after lost signal to thread 13357
I20260924 00:22:24.239224 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:22:24.239691 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:22:29.243309 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:22:29.243745 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:22:31.989804 13865 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:22:31.990281 13865 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:22:31.991958 13865 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:22:31.994043 13309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260924 00:22:31.997469 13309 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913): Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
W20260924 00:22:31.998006 13306 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
I20260924 00:22:31.998301 13306 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dba2791dd9734eb4b84bbbf2f7d674b8; no voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39
I20260924 00:22:31.998824 13865 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 00:22:34.247339 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:22:34.247787 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:22:39.251099 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:22:39.251541 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:22:44.255378 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:22:44.255927 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260924 00:22:44.286101 12650 debug-util.cc:398] Leaking SignalData structure 0x7b0800040aa0 after lost signal to thread 12519
W20260924 00:22:44.286906 12650 debug-util.cc:398] Leaking SignalData structure 0x7b08000d7fc0 after lost signal to thread 12653
W20260924 00:22:46.122488 12801 heartbeater.cc:644] Failed to heartbeat to 127.10.4.125:46875 (129 consecutive failures): Network error: Failed to ping master at 127.10.4.125:46875: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
W20260924 00:22:46.233340 12948 heartbeater.cc:644] Failed to heartbeat to 127.10.4.126:37913 (115 consecutive failures): Network error: Failed to ping master at 127.10.4.126:37913: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
W20260924 00:22:46.610198 12656 heartbeater.cc:644] Failed to heartbeat to 127.10.4.126:37913 (115 consecutive failures): Network error: Failed to ping master at 127.10.4.126:37913: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
I20260924 00:22:49.258689 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:22:49.259093 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:22:49.440776 14088 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:22:49.441236 14088 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:22:49.443252 14088 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:22:49.444447 13309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260924 00:22:49.447213 13309 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913): Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
W20260924 00:22:49.447710 13306 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
I20260924 00:22:49.447968 13306 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dba2791dd9734eb4b84bbbf2f7d674b8; no voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39
I20260924 00:22:49.448487 14088 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 00:22:54.262395 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:22:54.262952 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260924 00:22:59.236790 12796 debug-util.cc:398] Leaking SignalData structure 0x7b08000d3f40 after lost signal to thread 12665
W20260924 00:22:59.237690 12796 debug-util.cc:398] Leaking SignalData structure 0x7b08000da600 after lost signal to thread 12799
I20260924 00:22:59.265631 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:22:59.266163 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:22:59.631896 14192 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:22:59.632357 14192 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:22:59.634290 14192 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:22:59.635816 13309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260924 00:22:59.638758 13309 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913): Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
W20260924 00:22:59.639410 13306 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
I20260924 00:22:59.639655 13306 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dba2791dd9734eb4b84bbbf2f7d674b8; no voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39
I20260924 00:22:59.640205 14192 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 00:23:04.269279 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:23:04.269683 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:23:09.272161 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:23:09.272532 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260924 00:23:10.224709 12796 debug-util.cc:398] Leaking SignalData structure 0x7b08000d7bc0 after lost signal to thread 12665
W20260924 00:23:10.225637 12796 debug-util.cc:398] Leaking SignalData structure 0x7b08000dfe60 after lost signal to thread 12799
I20260924 00:23:11.338320 14325 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:23:11.338778 14325 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:23:11.340492 14325 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:23:11.342490 13309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260924 00:23:11.344794 13306 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
W20260924 00:23:11.345216 13309 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913): Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
I20260924 00:23:11.345510 13309 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dba2791dd9734eb4b84bbbf2f7d674b8; no voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39
I20260924 00:23:11.346007 14325 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 00:23:14.275563 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:23:14.276026 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:23:19.278786 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:23:19.279198 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260924 00:23:21.432917 12650 debug-util.cc:398] Leaking SignalData structure 0x7b08000e0ca0 after lost signal to thread 12519
W20260924 00:23:21.433789 12650 debug-util.cc:398] Leaking SignalData structure 0x7b08000dbde0 after lost signal to thread 12653
I20260924 00:23:24.282316 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:23:24.282737 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260924 00:23:26.035912 13354 debug-util.cc:398] Leaking SignalData structure 0x7b0800052ac0 after lost signal to thread 13280
W20260924 00:23:26.036523 13354 debug-util.cc:398] Leaking SignalData structure 0x7b08000b9ce0 after lost signal to thread 13357
I20260924 00:23:29.323310 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:23:29.323645 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:23:30.286293 14531 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:23:30.286720 14531 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:23:30.288540 14531 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:23:30.290565 13309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260924 00:23:30.292878 13306 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
W20260924 00:23:30.293304 13309 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913): Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
I20260924 00:23:30.293573 13309 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dba2791dd9734eb4b84bbbf2f7d674b8; no voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39
I20260924 00:23:30.294054 14531 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260924 00:23:32.802367 12940 debug-util.cc:398] Leaking SignalData structure 0x7b08000df520 after lost signal to thread 12812
W20260924 00:23:32.803244 12940 debug-util.cc:398] Leaking SignalData structure 0x7b08000705e0 after lost signal to thread 12943
I20260924 00:23:34.326835 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:23:34.327256 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260924 00:23:38.411652 12940 debug-util.cc:398] Leaking SignalData structure 0x7b0800040ac0 after lost signal to thread 12812
W20260924 00:23:38.412492 12940 debug-util.cc:398] Leaking SignalData structure 0x7b08000e04a0 after lost signal to thread 12943
I20260924 00:23:38.642576 14654 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:23:38.643185 14654 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:23:38.645466 14654 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:23:38.647902 13309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260924 00:23:38.650925 13306 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
W20260924 00:23:38.651350 13309 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913): Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
I20260924 00:23:38.651646 13309 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dba2791dd9734eb4b84bbbf2f7d674b8; no voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39
I20260924 00:23:38.652185 14654 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 00:23:39.346282 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:23:39.346633 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260924 00:23:40.063248 13354 debug-util.cc:398] Leaking SignalData structure 0x7b08000a1be0 after lost signal to thread 13280
W20260924 00:23:40.063764 13354 debug-util.cc:398] Leaking SignalData structure 0x7b080009d1e0 after lost signal to thread 13357
I20260924 00:23:44.349810 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:23:44.350221 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260924 00:23:46.497197 12801 heartbeater.cc:644] Failed to heartbeat to 127.10.4.125:46875 (249 consecutive failures): Network error: Failed to ping master at 127.10.4.125:46875: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
W20260924 00:23:46.596105 12945 heartbeater.cc:644] Failed to heartbeat to 127.10.4.125:46875 (246 consecutive failures): Network error: Failed to ping master at 127.10.4.125:46875: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
W20260924 00:23:46.991852 12656 heartbeater.cc:644] Failed to heartbeat to 127.10.4.126:37913 (235 consecutive failures): Network error: Failed to ping master at 127.10.4.126:37913: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
I20260924 00:23:47.897626 14743 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:23:47.898061 14743 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:23:47.899652 14743 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:23:47.901463 13309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260924 00:23:47.904012 13306 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
W20260924 00:23:47.904464 13309 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913): Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
I20260924 00:23:47.904736 13309 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dba2791dd9734eb4b84bbbf2f7d674b8; no voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39
I20260924 00:23:47.905265 14743 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 00:23:49.352727 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:23:49.353125 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:23:54.213826 13312 maintenance_manager.cc:419] P dba2791dd9734eb4b84bbbf2f7d674b8: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033357
I20260924 00:23:54.293949 13311 maintenance_manager.cc:643] P dba2791dd9734eb4b84bbbf2f7d674b8: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.079s user 0.051s sys 0.009s Metrics: {"bytes_written":10188,"cfile_init":1,"drs_written":1,"lbm_read_time_us":282,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2867,"lbm_writes_lt_1ms":30,"peak_mem_usage":0,"rows_written":16,"thread_start_us":425,"threads_started":1,"wal-append.queue_time_us":742}
I20260924 00:23:54.296268 13312 maintenance_manager.cc:419] P dba2791dd9734eb4b84bbbf2f7d674b8: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 943 bytes on disk
I20260924 00:23:54.297886 13311 maintenance_manager.cc:643] P dba2791dd9734eb4b84bbbf2f7d674b8: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":232,"lbm_reads_lt_1ms":4}
I20260924 00:23:54.356226 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:23:54.356622 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:23:59.359500 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:23:59.359851 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:24:02.140386 14958 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:24:02.140928 14958 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:24:02.142676 14958 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:24:02.144376 13309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260924 00:24:02.147483 13309 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913): Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
W20260924 00:24:02.147980 13306 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
I20260924 00:24:02.148270 13306 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dba2791dd9734eb4b84bbbf2f7d674b8; no voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39
I20260924 00:24:02.148787 14958 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 00:24:04.363052 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:24:04.363471 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260924 00:24:04.499820 12796 debug-util.cc:398] Leaking SignalData structure 0x7b08000c0f40 after lost signal to thread 12665
W20260924 00:24:04.501133 12796 debug-util.cc:398] Leaking SignalData structure 0x7b08000ceba0 after lost signal to thread 12799
I20260924 00:24:05.957966 15028 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:24:05.958403 15028 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:24:05.960080 15028 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:24:05.965191 13309 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913): Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
W20260924 00:24:05.965731 13306 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
I20260924 00:24:05.966050 13306 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dba2791dd9734eb4b84bbbf2f7d674b8; no voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39
I20260924 00:24:05.966605 15028 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 00:24:07.388839 15056 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:24:07.389321 15056 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:24:07.390748 15056 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:24:07.392488 13309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260924 00:24:07.395069 13309 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913): Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
W20260924 00:24:07.395463 13306 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
I20260924 00:24:07.395759 13306 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dba2791dd9734eb4b84bbbf2f7d674b8; no voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39
I20260924 00:24:07.396291 15056 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 00:24:09.366112 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:24:09.366490 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:24:14.369378 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:24:14.369783 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:24:15.577937 15201 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:24:15.578423 15201 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:24:15.580159 15201 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:24:15.581998 13309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260924 00:24:15.584767 13309 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913): Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
W20260924 00:24:15.585857 13306 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
I20260924 00:24:15.586112 13306 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dba2791dd9734eb4b84bbbf2f7d674b8; no voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39
I20260924 00:24:15.586583 15201 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260924 00:24:15.750820 12650 debug-util.cc:398] Leaking SignalData structure 0x7b08000d4640 after lost signal to thread 12519
W20260924 00:24:15.751626 12650 debug-util.cc:398] Leaking SignalData structure 0x7b08000e1360 after lost signal to thread 12653
I20260924 00:24:19.372211 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:24:19.372596 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:24:21.318251 15307 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:24:21.318607 15307 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:24:21.319998 15307 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:24:21.321739 13309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260924 00:24:21.323839 13306 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
W20260924 00:24:21.324144 13309 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913): Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
I20260924 00:24:21.324409 13309 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dba2791dd9734eb4b84bbbf2f7d674b8; no voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39
I20260924 00:24:21.324888 15307 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 00:24:24.375696 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:24:24.376091 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:24:29.378762 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:24:29.379086 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:24:34.382158 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:24:34.382613 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:24:38.887872 15607 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:24:38.888301 15607 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:24:38.890326 15607 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:24:38.892827 13309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260924 00:24:38.898787 13309 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913): Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
W20260924 00:24:38.899271 13306 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
I20260924 00:24:38.899537 13306 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dba2791dd9734eb4b84bbbf2f7d674b8; no voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39
I20260924 00:24:38.900063 15607 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 00:24:39.384577 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:24:39.384927 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:24:44.388017 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:24:44.388415 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:24:44.568060 15669 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:24:44.568524 15669 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:24:44.570636 15669 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:24:44.571592 13309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260924 00:24:44.574488 13309 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913): Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
W20260924 00:24:44.575448 13306 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
I20260924 00:24:44.575698 13306 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dba2791dd9734eb4b84bbbf2f7d674b8; no voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39
I20260924 00:24:44.576236 15669 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260924 00:24:46.835155 12801 heartbeater.cc:644] Failed to heartbeat to 127.10.4.125:46875 (369 consecutive failures): Network error: Failed to ping master at 127.10.4.125:46875: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
W20260924 00:24:46.939100 12948 heartbeater.cc:644] Failed to heartbeat to 127.10.4.126:37913 (355 consecutive failures): Network error: Failed to ping master at 127.10.4.126:37913: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
W20260924 00:24:47.329752 12656 heartbeater.cc:644] Failed to heartbeat to 127.10.4.126:37913 (355 consecutive failures): Network error: Failed to ping master at 127.10.4.126:37913: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
I20260924 00:24:49.390939 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:24:49.391263 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:24:51.178346 15742 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:24:51.178740 15742 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:24:51.180241 15742 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:24:51.182075 13309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260924 00:24:51.184593 13306 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
W20260924 00:24:51.185024 13309 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913): Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
I20260924 00:24:51.185295 13309 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dba2791dd9734eb4b84bbbf2f7d674b8; no voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39
I20260924 00:24:51.185781 15742 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 00:24:54.394340 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:24:54.394721 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:24:59.397374 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:24:59.397732 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:25:04.400800 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:25:04.401274 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260924 00:25:08.665548 12796 debug-util.cc:398] Leaking SignalData structure 0x7b08000dc700 after lost signal to thread 12665
W20260924 00:25:08.666417 12796 debug-util.cc:398] Leaking SignalData structure 0x7b08000df880 after lost signal to thread 12799
I20260924 00:25:09.403957 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:25:09.404301 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:25:10.998620 16037 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:25:10.999066 16037 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:25:11.001060 16037 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:25:11.002997 13309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260924 00:25:11.005546 13306 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
W20260924 00:25:11.005950 13309 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913): Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
I20260924 00:25:11.006240 13309 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dba2791dd9734eb4b84bbbf2f7d674b8; no voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39
I20260924 00:25:11.006798 16037 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 00:25:14.407529 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:25:14.407943 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:25:17.563375 16151 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:25:17.563817 16151 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:25:17.565428 16151 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:25:17.567246 13309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260924 00:25:17.569648 13309 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913): Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
W20260924 00:25:17.570061 13306 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
I20260924 00:25:17.570326 13306 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dba2791dd9734eb4b84bbbf2f7d674b8; no voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39
I20260924 00:25:17.570885 16151 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 00:25:19.410612 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:25:19.410972 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:25:23.593956 16253 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:25:23.594434 16253 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:25:23.596117 16253 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:25:23.598268 13309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260924 00:25:23.601455 13309 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913): Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
W20260924 00:25:23.601894 13306 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
I20260924 00:25:23.602159 13306 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dba2791dd9734eb4b84bbbf2f7d674b8; no voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39
I20260924 00:25:23.602689 16253 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 00:25:24.413904 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:25:24.414345 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260924 00:25:25.996356 12650 debug-util.cc:398] Leaking SignalData structure 0x7b08000460a0 after lost signal to thread 12519
W20260924 00:25:25.997246 12650 debug-util.cc:398] Leaking SignalData structure 0x7b08000d6b00 after lost signal to thread 12653
W20260924 00:25:26.624792 13354 debug-util.cc:398] Leaking SignalData structure 0x7b08000b5da0 after lost signal to thread 13280
W20260924 00:25:26.625404 13354 debug-util.cc:398] Leaking SignalData structure 0x7b08000b0d60 after lost signal to thread 13357
I20260924 00:25:29.434231 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:25:29.434576 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:25:34.437686 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:25:34.438092 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:25:36.817303 16475 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:25:36.817735 16475 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:25:36.819279 16475 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:25:36.821429 13309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260924 00:25:36.824044 13306 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
W20260924 00:25:36.824469 13309 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913): Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
I20260924 00:25:36.824765 13309 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dba2791dd9734eb4b84bbbf2f7d674b8; no voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39
I20260924 00:25:36.825361 16475 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260924 00:25:37.059475 12940 debug-util.cc:398] Leaking SignalData structure 0x7b08000d9920 after lost signal to thread 12812
W20260924 00:25:37.060302 12940 debug-util.cc:398] Leaking SignalData structure 0x7b08000d9a40 after lost signal to thread 12943
I20260924 00:25:39.440887 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:25:39.441265 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:25:44.444377 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:25:44.444787 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260924 00:25:47.180550 12801 heartbeater.cc:644] Failed to heartbeat to 127.10.4.125:46875 (489 consecutive failures): Network error: Failed to ping master at 127.10.4.125:46875: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
W20260924 00:25:47.277268 12945 heartbeater.cc:644] Failed to heartbeat to 127.10.4.125:46875 (486 consecutive failures): Network error: Failed to ping master at 127.10.4.125:46875: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
W20260924 00:25:47.674485 12655 heartbeater.cc:644] Failed to heartbeat to 127.10.4.125:46875 (489 consecutive failures): Network error: Failed to ping master at 127.10.4.125:46875: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
I20260924 00:25:49.447175 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:25:49.447510 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:25:54.450536 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:25:54.450953 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:25:54.965263 16754 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:25:54.965714 16754 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:25:54.967748 16754 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:25:54.969728 13309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260924 00:25:54.972122 13306 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
W20260924 00:25:54.972484 13309 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913): Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
I20260924 00:25:54.972782 13309 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dba2791dd9734eb4b84bbbf2f7d674b8; no voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39
I20260924 00:25:54.973340 16754 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260924 00:25:58.554355 12650 debug-util.cc:398] Leaking SignalData structure 0x7b08000da160 after lost signal to thread 12519
W20260924 00:25:58.555290 12650 debug-util.cc:398] Leaking SignalData structure 0x7b08000d5e40 after lost signal to thread 12653
I20260924 00:25:59.453869 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:25:59.454264 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:26:01.602746 16837 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:26:01.603111 16837 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:26:01.604728 16837 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:26:01.606098 13309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260924 00:26:01.608578 13309 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913): Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
W20260924 00:26:01.608932 13306 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
I20260924 00:26:01.609233 13306 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dba2791dd9734eb4b84bbbf2f7d674b8; no voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39
I20260924 00:26:01.609719 16837 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260924 00:26:02.547649 12796 debug-util.cc:398] Leaking SignalData structure 0x7b08000e31a0 after lost signal to thread 12665
W20260924 00:26:02.548493 12796 debug-util.cc:398] Leaking SignalData structure 0x7b08000c2f60 after lost signal to thread 12799
I20260924 00:26:04.457374 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:26:04.457804 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:26:09.460482 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:26:09.460846 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:26:11.068048 16995 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:26:11.068500 16995 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:26:11.070073 16995 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:26:11.072204 13309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260924 00:26:11.075202 13309 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913): Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
W20260924 00:26:11.075691 13306 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
I20260924 00:26:11.076035 13306 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dba2791dd9734eb4b84bbbf2f7d674b8; no voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39
I20260924 00:26:11.076700 16995 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 00:26:14.464164 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:26:14.464656 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:26:19.467103 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:26:19.467434 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:26:24.470782 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:26:24.471181 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:26:26.527947 17257 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:26:26.528440 17257 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:26:26.530563 17257 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:26:26.532480 13309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260924 00:26:26.534900 13306 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
W20260924 00:26:26.535377 13309 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913): Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
I20260924 00:26:26.535655 13309 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dba2791dd9734eb4b84bbbf2f7d674b8; no voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39
I20260924 00:26:26.536194 17257 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 00:26:29.473779 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:26:29.474171 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:26:34.118777 17386 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:26:34.119277 17386 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:26:34.120919 17386 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:26:34.123297 13309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260924 00:26:34.126024 13306 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
W20260924 00:26:34.126485 13309 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913): Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
I20260924 00:26:34.126783 13309 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dba2791dd9734eb4b84bbbf2f7d674b8; no voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39
I20260924 00:26:34.127396 17386 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 00:26:34.476403 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:26:34.476792 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:26:38.743520 17461 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:26:38.743914 17461 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:26:38.745636 17461 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:26:38.749903 13309 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913): Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
W20260924 00:26:38.750326 13306 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
I20260924 00:26:38.750603 13306 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dba2791dd9734eb4b84bbbf2f7d674b8; no voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39
I20260924 00:26:38.751071 17461 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 00:26:39.479075 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:26:39.479416 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:26:44.482594 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:26:44.483001 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:26:44.767249 17540 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:26:44.767799 17540 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:26:44.769560 17540 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:26:44.771440 13309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260924 00:26:44.783005 13309 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913): Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
W20260924 00:26:44.783582 13306 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
I20260924 00:26:44.783841 13306 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dba2791dd9734eb4b84bbbf2f7d674b8; no voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39
I20260924 00:26:44.784354 17540 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260924 00:26:47.506417 12802 heartbeater.cc:644] Failed to heartbeat to 127.10.4.126:37913 (593 consecutive failures): Network error: Failed to ping master at 127.10.4.126:37913: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
W20260924 00:26:47.625941 12948 heartbeater.cc:644] Failed to heartbeat to 127.10.4.126:37913 (595 consecutive failures): Network error: Failed to ping master at 127.10.4.126:37913: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
W20260924 00:26:48.018477 12655 heartbeater.cc:644] Failed to heartbeat to 127.10.4.125:46875 (609 consecutive failures): Network error: Failed to ping master at 127.10.4.125:46875: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
I20260924 00:26:48.549405 17599 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:26:48.549777 17599 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3a72a9440740a4948a000656ccabdf" member_type: VOTER last_known_addr { host: "127.10.4.126" port: 37913 } } peers { permanent_uuid: "fb563a88aa00432ba12db382b1435a39" member_type: VOTER last_known_addr { host: "127.10.4.125" port: 46875 } } peers { permanent_uuid: "dba2791dd9734eb4b84bbbf2f7d674b8" member_type: VOTER last_known_addr { host: "127.10.4.124" port: 45407 } }
I20260924 00:26:48.551136 17599 leader_election.cc:288] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913), fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875)
W20260924 00:26:48.555148 13309 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3a72a9440740a4948a000656ccabdf (127.10.4.126:37913): Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111)
W20260924 00:26:48.555528 13306 leader_election.cc:334] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fb563a88aa00432ba12db382b1435a39 (127.10.4.125:46875): Network error: Client connection negotiation failed: client connection to 127.10.4.125:46875: connect: Connection refused (error 111)
I20260924 00:26:48.555807 13306 leader_election.cc:302] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dba2791dd9734eb4b84bbbf2f7d674b8; no voters: 3c3a72a9440740a4948a000656ccabdf, fb563a88aa00432ba12db382b1435a39
I20260924 00:26:48.556309 17599 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dba2791dd9734eb4b84bbbf2f7d674b8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 00:26:49.485558 13275 master_runner.cc:186] Error getting master registration for 127.10.4.126:37913: Network error: Client connection negotiation failed: client connection to 127.10.4.126:37913: connect: Connection refused (error 111),
I20260924 00:26:49.485906 13275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:26:50.193225 13312 maintenance_manager.cc:419] P dba2791dd9734eb4b84bbbf2f7d674b8: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 943 bytes on disk
I20260924 00:26:50.204581 13311 maintenance_manager.cc:643] P dba2791dd9734eb4b84bbbf2f7d674b8: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.010s sys 0.000s Metrics: {"dirs.queue_time_us":836,"dirs.run_cpu_time_us":266,"dirs.run_wall_time_us":267,"thread_start_us":1123,"threads_started":2,"wal-append.queue_time_us":896}
W20260924 00:26:50.946264 12957 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.10.4.126:37913,127.10.4.125:46875,127.10.4.124:45407, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.10.4.126:37913,127.10.4.125:46875,127.10.4.124:45407, num_attempts: 1)
W20260924 00:26:50.946807 12957 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.10.4.126:37913,127.10.4.125:46875,127.10.4.124:45407, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.10.4.126:37913,127.10.4.125:46875,127.10.4.124:45407, num_attempts: 1)
W20260924 00:26:50.947072 12957 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.10.4.126:37913,127.10.4.125:46875,127.10.4.124:45407, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.10.4.126:37913,127.10.4.125:46875,127.10.4.124:45407, num_attempts: 1)
W20260924 00:26:50.947383 12957 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.10.4.126:37913,127.10.4.125:46875,127.10.4.124:45407, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.10.4.126:37913,127.10.4.125:46875,127.10.4.124:45407, num_attempts: 1)
F20260924 00:26:50.958763 12965 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e43f7024e9b745758c0915ae7605e671 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.10.4.126:37913,127.10.4.125:46875,127.10.4.124:45407, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.10.4.126:37913,127.10.4.125:46875,127.10.4.124:45407, num_attempts: 1)
*** Check failure stack trace: ***
F20260924 00:26:50.961323 12964 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b34b7e9f7270468b93a28c7329bbec21 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.10.4.126:37913,127.10.4.125:46875,127.10.4.124:45407, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.10.4.126:37913,127.10.4.125:46875,127.10.4.124:45407, num_attempts: 1)
*** Check failure stack trace: ***
*** Aborted at 1790209610 (unix time) try "date -d @1790209610" if you are using GNU date ***
PC: @ 0x0 (unknown)
*** SIGABRT (@0x3e800002811) received by PID 10257 (TID 0x7f61badfb700) from PID 10257; stack trace: ***
@ 0x2b0af0 __tsan::CallUserSignalHandler() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1915
@ 0x2b31e0 rtl_sigaction() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:2005
@ 0x7f61cc933980 (unknown) at ??:0
@ 0x7f61c7bf0e87 gsignal at ??:0
@ 0x7f61c7bf27f1 abort at ??:0
@ 0x2b4f37 __interceptor_abort at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:263
@ 0x7f61cac24df4 google::LogMessage::SendToLog() at ??:0
@ 0x7f61cac259a0 google::LogMessage::Flush() at ??:0
@ 0x7f61cac2aadb google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x3572c4 kudu::MasterStressTest::CreateTableThread() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/master-stress-test.cc:?
@ 0x3874ce _ZNSt3__18__invokeIMN4kudu16MasterStressTestEFvvEPNS1_26MasterStressTest_Test_TestEJEvEEDTcldsdeclsr3std3__1E7forwardIT0_Efp0_Efp_spclsr3std3__1E7forwardIT1_Efp1_EEEOT_OS7_DpOS8_ at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/type_traits:3840
@ 0x387381 _ZNSt3__116__thread_executeINS_10unique_ptrINS_15__thread_structENS_14default_deleteIS2_EEEEMN4kudu16MasterStressTestEFvvEJPNS6_26MasterStressTest_Test_TestEEJLm2EEEEvRNS_5tupleIJT_T0_DpT1_EEENS_15__tuple_indicesIJXspT2_EEEE at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/thread:281
@ 0x386acb std::__1::__thread_proxy<>() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/thread:?
@ 0x2b11b0 __tsan_thread_start_func at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:959
@ 0x7f61cc9286db start_thread at ??:0
F20260924 00:26:50.961323 12964 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b34b7e9f7270468b93a28c7329bbec21 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.10.4.126:37913,127.10.4.125:46875,127.10.4.124:45407, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.10.4.126:37913,127.10.4.125:46875,127.10.4.124:45407, num_attempts: 1)F000F20260924 00:26:51.003976 12962 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4a8f7937d4ef462592a1c80eefab1d0d on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.10.4.126:37913,127.10.4.125:46875,127.10.4.124:45407, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.10.4.126:37913,127.10.4.125:46875,127.10.4.124:45407, num_attempts: 1)
*** Check failure stack trace: ***
@ 0x7f61c7cd361f clone at ??:0